Introduction: Data as the “new oil”

Business leaders and technologists alike have embraced the immense value of information, particularly Big Data, heralding the phrase “Data is the new oil” in various contexts. We explore this concept as we continue to build the case for investing in data quality.

This observation stems from the undeniable reality that information volumes are experiencing exponential growth and that new technology advancements make it possible to store and analyse this data.

New possibilities for insights

The pundits assert with conviction, “Surely, with such a wealth of insights available, massive commercial benefits can be attained – for marketing, sales, and research and development.” However, the practical implications of poor data quality are already significantly impacting businesses’ operational capabilities.

Challenges of data silos

Consider this simple example: many corporations operate as conglomerates of business units, each maintaining their own systems, data stores, and processes. Consequently, customer master and behavioural records often become fragmented across different business units, product systems, or accounts.

This fragmented approach, often called data silos, creates a formidable challenge in achieving an accurate, enterprise-wide view of customer spending habits, overall value, or even consistent information such as telephone numbers or addresses. Clients may update one system with recent details but neglect to update others, making it even more challenging to comprehend customer behaviour.

It comes as no surprise, then, that companies struggle to gain a precise understanding of customer behaviour despite investing millions in business analytics and reporting tools. Unfortunately, the problem is compounded when large volumes of poor-quality information are added to this already complex situation.

Two Data Quality goals for valuable big data

To extract value from Big Data, companies must strive to achieve two critical data quality goals.

1. Integrate related data

Firstly, they need to establish connections between client data held internally in traditional data stores like CRM or billing systems and client data found in Big Data sources like social media. For instance, short-term insurance companies can utilize driving patterns derived from vehicle tracking systems or mobile phones to better assess the risk profiles of individual drivers. However, this can only be accomplished if the link between the driver and the vehicle is clearly understood.

2. Filter or improve poor-quality data

Secondly, companies must filter out poor quality and irrelevant sources from their feeds, thereby reducing volumes to manageable sizes. Failure to do so may lead to overwhelming management and storage costs. In the case of the insurance company mentioned earlier, there is no need to be concerned about the driving behaviour of an uninsured third party.

A strategic approach to data quality

Throughout history, organizations have grappled with managing data quality, even without the added complexities of emerging data sources. An enterprise Data Governance strategy should incorporate a measurable plan to ensure data quality, mitigating the impact of Big Data on analytics and operations.

The Data Privacy Challenge

Big Data presents an additional challenge – companies must ensure that their pursuit of increased insight does not ignore data privacy legislation. With the rise of global data protection regulations, safeguarding the rights of individuals and companies from unauthorized use of personal records is paramount. Therefore, companies should design their Data Governance programs to encompass both the appropriate use of information and its quality.

Think about it: Why is clean data important? It’s not just about hygiene; it’s about ensuring the vitality of your operations.

But here’s the kicker: what are the costs of poor quality data? Ignoring this aspect could lead to unforeseen consequences.

In summary, high-quality enterprise data serves as an essential building block for successful analytics, whether involving Big Data or other sources.