Many executives say they don’trust their own analytics.
This doesn’t mean that they don’t want analytics they can trust.
Why don’t executives trust analytics
There are many reasons that executives (and other decision makers) don’t trust analytics.
Some of these may be linked to political agendas or bad habits.
But in many cases, a lack of trust is due to the disconnect that often exists between data analysts and business.
At the technical level, analytics results may be skewed by poor data quality; by ambiguous terminology; or by inconsistent lineage.
Data quality: Missing, inconsistent or invalid values in data can materially distort analyses.
For example, one company reporting a high level of compliance to BBBEE procurement targets failed to recognise that the field indicating BBBEE level was mostly not captured meaning that the report was only measuring compliance against fewer that 20% of suppliers.
Ambiguous terminology: Having different reports measuring the same value but with different definitions, calculations or aggregations.
For example, marketing may measure customers as any consumer who has interacted with the company, sales may measure a customer as any consumer that has placed an order, finance may measure customers as a consumer that has made a payment. These kind of disconnects immediately create disparities in calculations such as Sales per Customer.
Inconsistent lineage: Having different sources for different reports resulting in differing outcomes.
Sales measure sales volumes and commissions as per the CRM, Finance uses the ERP. Different answers to the same questions
In most cases a lack of trust is created due to the reality that different answers are supplied to the same questions without the context to understand why the results differ. Many of these concerns can be resolved through better documentation of terminology, lineage and data quality.
Very simply, data analysts are frequently relatively junior staff that have (some) technical skills but are not deeply versed in the business.
In many organisations, data scientists are frequently employed straight out of university, carrying impressive credentials in statistical analysis, R programming, and the like, but with very little thought given to their ability to understand the unique business context, or to communicate their findings in a business friendly way.
The rise of the data translator is one symptom of this disconnect.
Ultimately, deep business understanding can only be gained by exposure to the business over a period of several years.
What can we do?
Technology exists to address technical concerns around analytics.
Report score carding enhances trust by weighting reports based on the quality of underlying data, the completeness of definitions, and the understanding of lineage.
To address, business disconnects requires a re-evaluation of how we source our analytics staff.
Companies need to consider data analytics career paths for existing, long-time staff that show an aptitude for problem solving, that ask relevant questions and that have a good understanding of the business.
These staff can be augmented by externally sourced specialists, who may have stronger technical skills that will complement the team.