Top 5 excuses for bad data!

1.) Our data is clean!

There are certain questions that trigger an automated response. If a store assistant approaches and says, “Can I help you?” you are almost certain to respond, “No thanks, I’m just looking!” Similarly, if you ask whether data is dirty you will almost certainly get the immediate denial.

Unfortunately, denial is not a strategy for improvement. As a data management professional you need to be able to focus on business needs that are not being meet due to poor quality data. For example, you may ask the purchasing manager whether it is possible that suppliers are being paid more than once, due to invoices being linked to multiple (duplicate) supplier records.

In my experience, asking more specific questions is unlikely to trigger an immediate, emotional response. Specific problems can be isolated and rapidly addressed with a data quality audit followed, if necessary with a remediation process.

2.) Our business process will sort it out!

Yes, business process is critical to addressing data quality issues! But if it is working so well then why are errors still creeping in?

The reality is that most business processes are forced to sacrifice quality to deliver on cost and time constraints. Very few managers are prepared to take on additional staff to improve on data capture accuracy levels. Nor are they prepared to take action against successful sales people that are ignoring corporate standards for data capture.

Automation can significantly reduce data capture errors by identifying and rectifying these at the point of capture – for example by correcting common spelling mistakes, adding missing information or ensuring data is captured in the correct format and the correct field.

Automated data audits can ensure that critical data is suitable to business needs, identify areas where the business process is failing and enable remeditation.

3.) Our new ERP/ETL/MDM solution will solve the problem!

It is tempting to believe that one investment will solve a multitude of problems – and given the correct selection criteria it is possible that it shall.

In an interesting research piece, Bloor Research’s Philip Howard discussed the importance of doing a relevant evaluation. In many cases, the new solution has been selected on the basis of its ability to meet a specific business need – so data integration for the ETL tool, improved business processing and integration for the ERP and so on. This does not mean that it will meet your data quality needs!

Do you have the budgets and internal capabilities to define and build, from scratch, the complex business rules necessary to handle different data types, multiple languages, misfielded data, etc? Does the solution come with these?

Do you know what queries to run to identify common data quality issues?

Do you have a requirement to provide automated data quality scorecards or trend reports to measure improvement?

Should data quality rules be reusable across your enterprise – run in the ETL batch, called by the ERP GUI, embedded in the legacy COBOL CICS application?

Have you thought about your data quality requirement or was it an afterthought? If the solution proposed was not selected according to your data quality requirements then it will probably not solve the problem – after all the system you are replacing didn’t!

4.) The problem is too big (or we tried before and failed)!

Successful data improvement approaches recognise that:

  • Eat the elephant one bite at a time. Successful data quality initiatives start small and address specific tactical needs. At BT, for example, the data quality team started with specific tactical projects that established the business “case for industrial scale data management”.
  • Invest in a scalable solution that you won’t outgrow. According to Nigel Turner,  ex BT head of ICT customer management delivery, “Data cleansing and data quality are not synonymous. With data cleansing, you can reach a certain standard of accuracy but if you don’t have a way to maintain that, the problem will just come back.” The data quality solutions deployed in the initial tactical projects were sufficiently robust and flexible to be reused – for example in the call centre to improve real time call data capture.
  • A holistic approach is required including process improvement, appropriate technologies and change management. Even small data management projects will require business change (on an appropriate scale) if they are to realise the full benefit. Specialist skills are a big help in achieving the necessary buy in.

5.) We can’t afford it!

Data quality (and data governance) initiatives are a little like rehab – many organisations have to hit rock bottom before they will begin them. Yet, like a drug addict that can’t stop, not addressing data quality issues can have devastating consequences.

For example, a business that cannot invoice accurately, due to issues with billing data, will suffer more than just loss of income. It can become difficult to distinguish between legitimate issues and chancers seeking to delay legitimate payments – cash flow may suffer or legitimate debts may be discounted or written off.  Billing issues are a major cause of customer dissatisfaction, and if these cannot be resolved quickly lead lead to churn. Your business may fail its financial audit – with devastating effects on the share price.

As discussed, your data quality initiative needs to start small and show measurable ROI quickly.

Our methodology aims at 3 month cycles for improvement – analyse, deploy, measure.  For example, by identifying and addressing issues with their billing data a local municipality was able to cut their debtor’s book by nearly 50% – a significant step towards a clean audit – for very little cost.

Similarly, BT started with small, measurable projects that showed small but meaningful ROI in short periods. Over time BT realised nearly R7billion in quantified benefits as a result of improved data quality. Are you sure you can’t afford that kind of return.

6.) Yes I know I said 5

What are the common hurdles you find inhibiting your efforts to deliver quality data?

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.