The relentless churn and constant advances on both hardware and software landscape in IT domain often renders “the contemporary”, obsolete every 4-5 years. Organizations tend to add technology & platform components to the original estate to take advantage of topical advancements.
Additionally, corporations evolve both organically and inorganically making it necessary to adopt new processes and procedures to manage co-habitation and co-existence of business intent across erstwhile competing entities. To add to the milieu of complexities technology companies retire platforms and versions and withdraw support.
As an organization, how are you preparing to address this issue? Read on to understand some of the signals that can help you pro-actively handle the situation.
Data Estate Revaluation: When, How & Why!
Managing obsolescence is not a one-time activity. It is a constant problem that needs “Eyes & Ears” that closely watch the markers that help spot when the signal turns amber.
The need to strategically manage “end-of-life” of your IT landscape demands a systemic approach and is ideally done annually by a dedicated team, that are not part of traditional maintenance and support functions. An independent team is best suited to identify, mark, recommend & plan new technology adoptions. Interestingly there are both engineering & business pointers that help “spot the rot”.
Some Engineering Indicators For Evaluating Data Estate Are :
Figure 1 : Key Engineering Indicators for Evaluating Data Estate
- Gaps in Data Platform Capability:
While most OEMS provide the best of capabilities to manage Data Management challenges, to overcome feature / functionality limitations as well as to accommodate company specific business process and requirements, corporations tend to customize platforms intents. Re-assessing and preparing for de-customization programs coinciding with new major releases of an OEM platform will optimize usage of several components that make up the estate.
While OEMs fix issues for the first five years of purchase and provide security fixes for up to 10 years, it is a good practice to re-look & re-assess the efficacy of the estate components and architecture. If your data estate is approaching 4 years or older, you may want to consider modernizing your estate.
In addition to ensuring adaptation to latest advances this will also help prepare for unforeseen eventualities like the following example. A leading corporation was using Cognos Data Manager to transform and load data into Oracle Data Warehouse. However, when IBM acquired Data Manager and decided to end support for the product the Data Management team had to hastily research and look for a replacement.
- Availability of Information
Business stake holders can make reliable decisions if they have access to clean and integrated information. A recent survey published by McKinsey points out that the lack of availability of quality data causes employees to spend almost 29% of time on non-value-added tasks like access, clean, and work on data required for analysis.
Check if your data analysts are spending significant time in researching and cleaning data or are they able to use data with minimal time spent on re-engineering them.
- Access to Quality Data
A Data Health Research conducted in 2021 shows that 60% of business executives don’t always trust their company’s data.
Figure 2 : Time Spent In Gathering and Cleansing Data
“Gathering Data” & “Cleaning Data” are “Data Engineering” activities and presents the maximum possibility of control & automation. If your organizational spend at this stage consumes anything more than 15% of the total “Data Journey” time then, I am sure you would want to initiate investigation into your existing Data Estate.
Some symptoms that signal the need for a re-assessment are
- Increasing maintenance costs: Frequent questions & support requests from business to correct data.
- Customer dissatisfaction & retention: Delayed responses to queries & distorted metrices.
- Reputational damages & fines due to compliance failures.
- Missed opportunities: Inaccurate analysis due to data decay.
- Regulatory Compliance
Businesses tend to be laggards when it comes to following compliance norms given the pace at which regulatory norms change or get enhanced.
A closely knit international business community necessitates frequent assessment of the company’s compliance readiness be it related to cybersecurity, data, legal and other domain specific norms.
Does your firm follow a governance model that regularly assesses the impacts of changes in compliance guidelines?
New regulations like GDPR (Europe), Personal Information Protection and Electronic Documents Act (Canada), The Consumer Privacy Act of 2018 (California) and others are all connected to requirement “traceability.”
The GDPR allows the EU’s data protection authorities to levy fines of up to €20 million (roughly $2,372,000) or 4% of global turnover for the previous financial year, whichever is greater.
Additionally regulatory compliance requirements impose increased transparency obligations on businesses in terms of data tracking and auditing.
A data estate evaluation can be used to verify compliance to data governance best practices with respect to various regulations, simplify the complexity of often inter-linked volumes of data through lineage discovery to gain visibility into the data flow.
- Slower Speed and Performance of Data Platform
An IDC survey found that 76% of respondents said their inability to analyse current data hinders their capacity to take advantage of business prospects. 86.5% of organizations utilize ETL to shift at least 25% of their data between transactional and analytical systems. Furthermore, almost 64% of ETL data that reaches an analytics database is at least five days old.
For most organizations, this is a major roadblock in their efforts to provide the best client experience at the appropriate time.
- Changing Business Demands
Today, companies are expanding globally at a much faster pace than a decade ago. Mergers and acquisitions are adding complexity to the mix. In such times, cross platform data sharing that may even have legacy systems, pose innumerable challenges to sharing relevant data within desirable timelines.
Scattered data will be detrimental to the modern business landscape and this demands re-evaluation of the data management estate.
- High Licensing Cost and IT Spends
With the constant strain on the annual licensing fees and limited IT budgets, the company’s digital management budget is strained. Automation & compulsive documentation is key to ensuring optimal functionality and upkeep of one’s systems.
A top insurance corporation from Europe, employs more than 15 consultants, 30-40 person-days to complete scheduled data warehouse updates and releases. Additionally, every quarter 10-15 days was spent to support the DWH when enhancements were pushed to production on one of the enterprise core systems.
The company spent lot of bandwidth in ensuring their data warehouse kept track of changes in the core systems compromising time for innovation.
- Very Large Maintenance Teams
Another sure shot indicator that a corporation’s estate needs a complete re-look is the size of the team that operates the maintenance and upkeep of the estate. I have seen disproportionally large IT teams being demanded to support existing systems. An end 2019 survey mentions, a minimum of 20-30% of all work related to data management can be automated.
Organizations need to start looking into ways to modernize and find cost-effective solutions to meet their digital requirements.
Automated data warehousing and no code data platforms like Ab Initio can ensure lesser spending on IT support.
Some Business Metrices to Look For Are :
Figure 3 : Business Metrices for Evaluating Data Estate
- Lost Revenue Due to Data Unavailability
Legacy data estate can lead to lost revenue in many ways. Take, for example, communications that fail to convert to sales because the underlying data estate is incapable of implementing necessary modifications to enable quick changes to business rules.
- Loss of Operational Efficiency
Modern business processes use data warehouses or more contemporary systems like Data Lakes or Lake houses to bring together operational metrics from different source systems in real time.
In a manufacturing facility, for example, data is collected and sent to a data warehouse, where it is combined with information on materials supply, product quality, and outbound logistics, to create the dashboards that facility managers use. Most often there is a mix of applications and technologies which tend to compromise efficiency as time progresses.
Problems cannot be detected or corrected in real time if updates from source systems are hindered due to performance issues.
- Breaches in Data Security
For failing to clearly demonstrate the security measures it implemented to secure the data of EU users, Meta Platforms (FACEBOOK) Ireland was fined €17 million by the Irish Data Protection Commission (DPC) in March 2022. This fine was related to reports of twelve personal data breaches discovered in 2018.
Above example shows the need to protect organizations from data losses and thefts. A modern data estate along with stringent governance norms will ensure that organizational information is secured.
Furthermore, security becomes increasingly important if your data contains personally identifiable information that must be carefully managed to comply with consumer protection laws.
- Disappointing M&A Integration
Up to 90% of merger & acquisition plans fail miserably. One of the biggest challenges arises due to data integration issues. Post-migration, companies often end up spending thousands of dollars in hiring data professionals to fix data quality problems.
While market & economic due diligence is a critical aspect it is equally important to investigate and assess the data estate of the target company to save oneself the heartbreak of failed merger.
The SWOT analysis of a prominent Insurance major in the US mentions the current structure has resulted in failure in the consolidation of various mergers aimed at vertical integration.
- Amount of IT Budget Spent in Paying Technical Debt
In a recent McKinsey survey, CIOs estimated that technical debt amounts to 20 to 40 percent of the value of their entire technology estate before depreciation. For large organizations this means spending hundreds of millions of dollars just in maintaining legacy applications and systems.
A company that spends more than half of its IT project budget on support and maintenance is going to find it difficult to allocate resources for innovations and new offerings.
- Penalties Due to Non-Compliance
If your firm is facing compliance issues, then your data estate and the engineering and governance aspects of data management must be critically looked at.
A prominent bank paid $400 Million to federal regulators due to its failure to establish effective risk management.
Data Governance issues typically manifests as failures in the areas of “Risk Management” & “Compliance”.
I am sure the bank’s management would either have strengthened their Data Governance programs or would possibly have re-assessed the efficacy of the platform they were using.
Conclusion: Resisting Change Can Be Taxing
Continuing with legacy systems and the lack of ability to determine what your data is telling you will be detrimental to your company’s growth. Analytics and transactional decisions are supported by a modern data platform, which also helps to reduce the overall cost of data infrastructure.
As an organization grows and expands, business demands increase. More data is generated; thus, the underlying systems, software, and hardware must be equipped to support easy access and analysis of data and use it to make informed business decisions.
At Elait, our consultants deliver innovative and scalable solutions across industry verticals that tackle challenges like data latency, high volume data processing, data integration, load & extract performance, and mass data transit & migration between systems, to name a few.
Our consultants can help you evaluate your data estate and gain actionable insights to enable the unification and integration of data across the digital ecosystem for digital extensibility, business scalability, agility, and responsiveness. Enhance Your Business Agility and Efficiency with Advanced Data Solutions.
Get In Touch With Our Experts.
RELATED ARTICLES
References:
https://query.prod.cms.rt.microsoft.com/cms/api/am/binary/RWPJce
https://www.abinitio.com/en/data-catalog-quality-governance/semantic-discovery/
https://www.dataversity.net/case-study-pally-biscuits-uses-modern-data-estate-to-enhance-decision-making/#
https://www.abinitio.com/en/customers-case-studies/
https://www.dataversity.net/case-study-venture-logistics-improves-approach-to-data-integration/
https://www.abinitio.com/en/data-catalog-quality-governance/semantic-discovery/
https://research.aimultiple.com/digital-transformationstats/#:~:text=Based%20on%20a%20survey%20of,reach%20%241.78%20trillion%20in%202022
https://www.moqdigital.com/insights/is-it-time-to-modernise-your-data-estate
https://research.aimultiple.com/digital-transformation-stats/#:~:text=Based%20on%20a%20survey%20of,reach%20%241.78%20trillion%20in%202022
https://www.ceotodaymagazine.com/2018/05/slow-data-kills-business/
https://dataladder.com/dont-acquire-a-company-until-you-evaluate-its-data-quality/
https://edition.cnn.com/2020/10/07/investing/citigroup-fine-risk-management/index.html
https://www.dataversity.net/case-study-helsana-automates-their-data-warehouse-infrastructure/
Rajeev heads business strategy at ELAIT. He has more than three decades experience in Program Management and has participated in platform and technology adoptions across many corporations. He writes articles addressing challenges faced by enterprises around data.