How Does Your Business Become Information-Driven and not Just Data-Driven?
By Martin Lindberg Lüttge, Director, Business Data Solutions, at ComputerWorld DataDriven Event, March 2023.
Today we’ll talk about how you can become an information-driven business. This involves:
- Set data free, so it can become information
- Think operational and analytical together
- Create the right data culture and data skillset
- Set a specific plan that talks into the strategy
Typically, data is divided into two separate tracks: operational and analytical.
Honestly, this approach presents challenges and a lack of synergies. If the data were treated as a single track, people could potentially derive more value from it.
When you set data free, it’s also important to make sure that your team knows how to handle it. That means we have to teach them:
- What it all means
- How they should use it
In addition, it’s essential to ensure that the use of your data not only sounds good on paper! It should also be directly linked to your company’s goals.
That’s the only way to go from a data-driven to becoming a truly information-driven business.
Let's Start by Setting the Data Scene
2010:
So let’s begin.
When I began my professional career in 2010, the use of data was already on the rise. Business Intelligence (BI) was starting to gain traction, though it was still primarily managed by IT and treated as an IT project. The concept of democratizing data was rarely discussed, and IT was primarily responsible for protecting data, with technical definitions being the norm.
Fair to say that truly information-driven businesses were an exception to the rule.
2015
Jumping to 2015:
Between 2010 and 2015, a lot transpired, and the conversation shifted toward self-service. It became essential for business users to have the ability to conduct their own analyses and work with data. While organizations discussed it, they had not yet made significant progress.
Many challenges surfaced, such as concerns about the potential risks associated with freeing up more data. While IT continued to be responsible for protecting the data, the definitions were now being managed by the business (and BI).
In 2015, some businesses were information-driven, while others were still in the process of transitioning towards a more data-focused approach. However, it’s important to note that the meaning of being an “information-driven business” had different meanings and implications in 2015 than it does today, as technological advancements and changing market conditions have led to an increased emphasis on data-driven decision-making in recent years.
2019
The concept of Data Mesh was introduced for the first time in 2019, but it has only gained significant traction now in 2023, about 4-5 years later. This has fuelled the democratization of data, as more companies are placing data and business intelligence within the business itself.
While IT still provides support, data, and BI are now primarily integrated into the organization and used by people in their daily work. As a result, data definitions and processes are now mostly owned by the business, with IT providing support.
Transitioning to an information-driven business model has become a top priority for many organizations.
Great!
2022
Last year, I witnessed the first successful rollouts of data mesh, where ownership and data competencies were present within businesses.
I’m so happy to see that more and more people use the power of data and analytics and become information-driven businesses that make smarter, more informed decisions at every level.
2023
As we step further into 2023, I’m excited about what lies ahead on the data side. I remain hopeful that there will be many positive developments for our data.
I encourage you to contribute to your business and advocate for the changes that will make a difference.
In today’s digital age, information is king, and companies that prioritize data-driven decision-making are the ones succeeding as information-driven businesses.
Now make it happen!
Information Maturity staircase
But how do you know how data-mature your company is?
To try to explain it, I came up with this Information Maturity Staircase. It illustrates where we are regarding data and how much we want to improve – which is crucial for any information-driven business.
We start at the bottom, where the data is in its original system and is only being used from here, which means no consolidation between systems. That’s the static data landscape.
Then we move to step 2, where the business starts to understand the data better and trust it more, but it’s still mostly in the hands of the IT department. They use things like data extracts and Excel sheets to get the info they need.
Step 3 is where the data starts to really come alive in the organization. At this stage, organizations have a BI solution, where people can find trusted information that’s good enough for the board room. Some data starts to flow automatically through systems or point-to-point integrations.
When we reach step 4, all the data is integrated into one platform, and it’s all available. This might be through a data hub, which works with our integration and BI solutions. For an information-driven business, this is a significant milestone that enables efficient access and analysis of data.
The last stage is where information is part of every decision we make in the business. Data products are a part of the data mesh that the organization has implemented to push the responsibility of data definitions and ownership into the domains where the knowledge of the data exists.
It’s where we’d love to be eventually, and it’s the ultimate goal of any information-driven business. Off course, the path to this isn’t without challenges, but it is possible.
What are the Typical Challenges in Becoming a Truly Information-Driven Business?
One of the biggest issues I encounter in my work with organizations is the increasing demand for data.
How is this an issue you might think?
Well, as businesses want to become more information-driven, there is a growing need to access data quickly and efficiently. But, it often takes too long to get the data.
- Organizations don’t have enough skilled data people to meet the demand or
People don’t know whom to ask for the data they need
It can be confusing and frustrating to figure out where to go to get the right information. Especially in large organizations with many different data sources.
To address this, it’s important to have a centralized approach to data management and a clear understanding of whom to ask for the data needed to make informed decisions.
The platform itself can also be a challenge. Organizations need to ensure they have a flexible architecture that supports their current and future needs, while also having clear ownership and governance over their data.
In other words:
It’s important to know who owns the data and where it should live!
Creating the right culture and governance around data means having the right teams in place to manage and analyze data and creating processes that ensure data is accurate, secure, and accessible. By building a strong foundation in these areas, you can better position yourself to become an information-driven business.
Finally, the biggest challenge is figuring out where to begin. It depends on the road you want to take as a business. However, it’s important to prioritize people and processes. These are the building blocks that will help you achieve your goals.
Building an Information-Driven Business: Prioritizing People, Processes, and Architecture
On the process side, it’s essential to have a central unit within the organization that can help identify the data needed to meet business demands. This could be a dedicated data team or a business intelligence unit.
In larger organizations, sometimes people end up working towards the same goal without even realizing it. It’s like being in a skyscraper with your team, all trying to reach the same endpoint at the same time, but you’re all on different floors and can’t see each other. You need to find a way to communicate and coordinate so you can reach the endpoint together.
Once you know what data you need and what you want to do with it, it’s time to operationalize it. You need to analyze the operational and analytical needs and find ways to bridge the gap between the integration and BI departments. This will help you utilize synergies and make the information available.
And finally, you need to get the data out live. This is where ownership and stewardship are key. You need dedicated stewards who can help find the right path to the data and train the business on how to access and use it effectively. They’ll also be responsible for quality assurance and control of who can see what.
Overall, becoming an information-driven business requires a concerted effort to prioritize people, processes, and architecture.
By focusing on these areas, your organization can better position itself to meet the growing demand for data and drive informed decision-making.
Let’s move on to the Architecture part:
Building an Information-Driven Business: Prioritizing People, Processes, and Architecture
Let’s start with the process part.
On the process side, it’s essential to have a central unit within the organization that can help identify the data needed to meet business demands. This could be a dedicated data team or a business intelligence unit.
In larger organizations, sometimes people end up working towards the same goal without even realizing it. It’s like being in a skyscraper with your team, all trying to reach the same endpoint at the same time, but you’re all on different floors and can’t see each other. You need to find a way to communicate and coordinate so you can reach the endpoint together.
Once you know what data you need and what you want to do with it, it’s time to operationalize it. You need to analyze the operational and analytical needs and find ways to bridge the gap between the integration and BI departments. This will help you utilize synergies and make the information available.
And finally, you need to get the data out live. This is where ownership and stewardship are key. You need dedicated stewards who can help find the right path to the data and train the business on how to access and use it effectively. They’ll also be responsible for quality assurance and control of who can see what.
Overall, becoming an information-driven business requires a concerted effort to prioritize people, processes, and architecture.
By focusing on these areas, your organization can better position itself to meet the growing demand for data and drive informed decision-making.
Let’s move on to the Architecture part:
Here you see a large overview of different reference components on the architecture side. You will most likely have some of it but not all of it. Building a solid architecture foundation is critical for becoming an information-driven business.
The operations side is on the left and the analytical part is on the right.
Let’s look at a specific example to see how these two sides can speak together.
We have a customer portal that customers can access through their mobile phones. When a customer logs in, an API call is sent to fetch data from both our CRM and ERP systems. The data is then sent back to the customer, and they can also update their information, such as adding their address.
This kind of information is valuable to us and can be shared with relevant stakeholders. For example, the sales manager can be notified when a big customer moves to a new address. We can easily capture and store this information in our data hub and warehouse for future use. While this process doesn’t require real-time data, it can happen relatively quickly to ensure our sales team has the information they need on time.
Here’s another example: imagine customers want to see the top ten products that sell the most.
If we have an old ERP system, we can’t make live requests to get the data. But we have another option.
We’ve already made sure that our sales manager can check which products are purchased the most. So, we’ve already got the data we need, we just need to make it operationally available too!
This means we can shoot the information back to our data hub, so when a customer logs in, the integration platform will compile the information from our CRM and ERP systems, plus some from our data warehouse via our data hub.
And that’s where the magic happens – the synergy!
But how do we Make Sure We’re Designing Things With the Information-Driven Business in Mind?
It’s crucial that our solutions aren’t just technical fixes to technical problems.
We need to focus on solving real business problems with technical solutions. Take this integration example, for instance. It’s not enough to just think about how to move data from one system to another.
We need to understand the whole business process behind it. Only then can we map out the flows and figure out which API’s and integration processes we need to use.
And it’s not just a matter of a technical developer sitting down with a database manager and figuring out how to make two systems talk to each other. The business side needs to be involved to make sure we’re designing with the business goals in mind. Only then should you take it back to the machine room and build it.
That way, you get a business-oriented design for your integrations.
On the data side, specifically in business intelligence (BI), we’ll need to create a conceptual model. This means defining what we mean by certain terms like “customer”. We need a common definition that everyone in the business can agree on. And sure, customers might come from different source systems.
But that’s not a big deal. We can handle that on the technical side later.
How is a Modern Data Platform Structured?
A modern data platform is typically structured as follows:
Source Systems: These are the various data systems used by the company, such as Microsoft Dynamics, SAP (maybe with ‘aXis for SAP’), Salesforce, etc.
Raw (or Staging) Layer: This is the layer where data is ingested 1:1 from the source systems to minimize load pressure on the sources. This layer should (today) be utilized by both integration and BI. Therefore this layer will be placed in the Data Hub. Traditionally, the Raw area would be located in the Data Warehouse (for BI purposes only).
Today that doesn’t make sense.
When we start to split this area up, we allow for more efficient and effective data sharing between systems.
Data Warehouse: This part consists of at least two layers, Curate and Serve.
Curate is where we ensure that data from the Raw layer in the Data Hub is filtered, cleaned, and transformed before it is sent to the next layer which could typically be the Server layer. This is where the cleaned and transformed data is now aggregated and made ready for serving either as Data as a Service or in an analysis solution like Power BI. It’s the place where the company maintains its corporate “one version of the truth.”
Data Products: This is where domain-specific data is stored. The data experts get ownership of the data and define how it should be modeled. The data is made visible to them in a raw format, and they can use it to create i.e. digital twins for their specific domain. The power to model and structure the data is here pushed to those who know the data best.
With this structure in a Modern Data Platform, we will enable the use of data both operationally and analytically, to become even more information-driven. Data can now flow both from and between sources and to the analysis solutions as well, but back through the Data Hub to the operational solutions as well.
This approach helps ensure that information can flow quickly throughout the organization, which is essential for making good decisions in real-time or near real-time.
By sharing the cleaned and transformed data 1:1 with the integration area, legacy systems that may not be API first or responsive can still access the data they need in a timely manner.
Yes, a collaboration between the operational and analytical sides of a company is crucial to the success of a modern data platform. We need a common understanding of why this is necessary.
The operational side is focused on using data to improve business processes and operations in real-time, while the analytical side is focused on using data to gain insights and make strategic decisions.
To build a modern data platform, that meets the needs of both sides, it’s important to have a shared understanding of the goals and objectives of a project.
What it all boils down to is:
open communication and collaboration between both integration and BI – the operational and analytical data planes.
It can seem quite trivial, but bringing both sides together, in the same room can do wonders. It can help to break down barriers and foster a common understanding of why a project is important and what the expected outcomes are.
By working together, the operational and analytical sides can ensure, that data is being collected, processed, and analyzed in a way that meets the needs of business processes and provides the information to make the right decisions at the same time. This will ultimately lead to more efficient and effective decision-making and will allow us to encompass all aspects of the data journey.
Steps to Achieving Information Maturity: What’s Required?
Let’s return to the Maturity ladder, I mentioned earlier.
What does it take to reach high data maturity in your company?
Step 1 – Is the starting point and a static landscape.
Step 2 – To reach step 2 of the data maturity ladder, you need to have at least a Data warehouse and a BI-solution in place to collect information.
Step 3 – Moving on to step 3, you’ll need a real organization focused on BI and data to ensure that any changes made in the business are anchored properly. This includes having a change management process in place and assigning someone to take ownership of it.
Step 4 – Once you’ve reached step 4, an integration platform should be in place to centralize system integrations and prevent changes in one system from affecting the entire chain. At this point, you can begin to establish a data hub.
Step 5 – When you reach the final stage, information should flow freely throughout the organization, with a data mesh in place.
Don’t be afraid of what might happen if you let it flow. Data will move around no matter what, so it’s important to decide whether you want to be part of that movement or not. To make it work, you need to have a clear process in place so people know whom to turn to and that someone is there to help them along the way.
Whether you’re on the operational or analytical side of the business, it’s important to see both sides as two parts of the same whole. This will prevent any build-ups on one side that can’t be supported by the other.
In terms of culture, it’s important to create awareness around data and how it can be accessed without being too heavy-handed.
When creating a roadmap, make sure to include specific activities so it doesn’t become a vague data strategy on top of a business strategy without any concrete initiatives.
Question from the audience.
“It sounds very easy. But we know it isn’t. What surprises people most in the process of becoming more information-driven in organizations? Is it the resources needed? Is it the organization that doesn’t know how to work together, or which kind of problems is it?”
Great questions!
Martin replies:
It may sound like a simple process, but in reality, becoming more information-driven in organizations can be challenging.
One common surprise for organizations is the number of people that need to be involved in the process from the very beginning. Simply having one IT architect and one member from top management is not enough. It’s important to hear from people around the organization to ensure that the initiatives are successful. You don’t need to gather 50 people around a table, but it’s essential to have representatives from the data/BI team, integration team, top management, and enterprise architecture. This way, you have all the relevant parties needed to ask the right questions that may affect the entire setup.
Another question:
“How many people are involved in the conceptual model to get it to work?”
Typically, the conceptual model will require domain-specific knowledge, such as sales or production. The workshop to develop the model usually takes half a day to one day and involves 6-8 people sitting around a table. It’s important not to have too many people involved, as that can lead to some participants feeling excluded or not having a say in the process. Once the workshop is completed, the involved individuals take the ideas and knowledge back to their respective business units to implement and refine the model further.
And that’s all for me.
Building an information-driven business requires more than just collecting and analyzing data. It requires collaboration across departments, a clear and concise process, and a culture that values data and its potential. It may not be an easy journey, but with the right mindset and approach, the rewards can be significant. Remember, it’s not about being data-driven, it’s about being information-driven.
Thank you.
You might also like this blog post:
Struggling to Get a Complete Overview of Your Data Landscape (16 Point Checklist)