Most data experts agree that 80% of analytics effort is in integrating data. Often this is indicated by pairing two Data Engineers with every Data Scientist. Even with this being the case, from data scientists to finance analysts and everyone in between, if you’re producing analysis and charts, the chances are you spend a lot of time crashing data together.
Why do organisations spend so much time and money integrating data?
What Do We Mean By ‘Integrated Data’?
Integrated data is data that has been taken from different sources and combined in a consistent way. Integrated data can enrich your analysis. For example, it can help you predict if your customers will like that brilliant new product or service you’re working on. If you work in a big company you probably have a data warehouse that contains integrated data, like the one shown in the image above.
Where Is It Used In The Enterprise? And Why Invest?
Let’s look at the chart in Figure 2 below
OK – there is a lot here to describe, so let’s break it down, layer by layer.
The chart is organised around different groups in an enterprise. You can think of a typical organisation being made up of individuals, teams, branches and departments which all roll up to the whole enterprise – shown in Figure 3:
As the size and scope of these groups grow, they need more context about the world around them to make decisions – or more integrated data.
Individuals need to understand their own performance, while enterprises need a broad, overall view as shown in Figure 4.
The control an individual has over the integrated data they work with diminishes as you move through the groups. Enterprise integrated dataminishing control has an inverse relationship to the amount of governance you need. Australian Privacy Principles, GDPR, HIPAA, and so-on means that everyone needs to understand their data obligations.
Figure 7 shows how the increase in governance & need for integrated data correlates to increased capital investment needed as more data is integrated. But the good news is that the cost per user falls rapidly as more people use integrated data, as illustrated in Figure 8, below.
Better Insights and Better Outcomes With Integrated Data
And this should translate into better insights, and better outcomes. The better you get at integrating data from different parts of the business, the more insight you can give to more people – and maybe make a big dent in that 80% we talked about at the beginning.
To find out more about the value of integrated data, or how Elait can help you with your integrated data management journey, please get in touch through our contact form.
Phil Watt is a Director for Elait Australia, specialising in data engineering and data governance solution services. He has more than 25 years’ experience in large-scale enterprise analytics and data engineering programmes, and his interest is in the journey to scale business programmes from small, proof of concept initiatives through to operational company-wide solutions with a high strategic impact.
Phil has deep experience in applying business analytics in the CME and FS sectors in Western Europe and South Pacific, including global technology leadership roles for Fortune 500 companies. He recently drafted the technology pillar of the Data Reform Strategy for the Victorian State Government and led its implementation in its first year.