The most dangerous disconnect in most IT projects are those we don’t realise we have. They are caused by assumptions that, because we use the same terms, we mean the same thing.
In many cases, terms that are used are, at best, loosely understood and, at worst, may have very different meanings for different business uses. For example, at one of our clients the term Recourse Business Unit is used by almost the entire business to identify the business unit that KYCd the client. For the credit risk team, however, this term is used to describe the business unit that is carrying the credit risk – in many cases a different BU.
This really becomes a problem when delivering enterprise wide projects, such as master data management or ERP, or when trying to reuse tactical solutions in other areas to reduce overall IT spend.
This problem is the driver for the various metadata initiatives that are being kicked off in some organisations. As discussed in my earlier post, I don’t like the word metadata. What is metadata anyway? In itself the term is ambigous and can lead to its own levels of confusion.
Some examples of critical business metadata may include:
A business terms dictionary – what do we mean when we talk about Contracts, or Profit? Is there a standard enterprise understanding or calculation – or do we need to cater for departmental uses? These are key questions that should be addressed by the data goverance team (if you have one) or by the project team if you don’t.
A data dictionary of key attributes holding information such as their Use, their lineage, their owner, as well as technical data quality rules such as whether they are required, or should be unique. Of particular interest, in my opinion, is the source of data. Do all users recognise the same source or do you have multiple versions of the truth creating inconsitency for key reports or metrics?
A set of data governance or data quality rules such as “A supplier record must include valid banking details to facilitate payment.” These are necessary in order to understand whether data is fit for purpose. They are also key indicators of broken business processes and for root cause analysis.
Technical metadata may include objects such as data flows and data models.
Organisations should focus on key business processes to identify which data elements require attention and documentation. Any organisation will be overwhlemed by the volumes and complexity if they try to address everything at once. Focus on the key processes and this will limit your problem to a small number of key indicators and a manageable number of data attributes and systems.
Finally, be cautious of significant investments in passive metadata – i.e. documentation that cannot be reused and will quickly get out of date. Where possible. metadata should be used actively – for example, data quality rules implemented in a data quality platform can provide dashboards and trending on the fitness for purpose of real data that can be adapted to address changes and can be acted upon to improve data quality.
So what is brown and sticky? Well I don’t know what you were thinking but the answer is a stick.
This post was originally published on the dataqualitymatters blog
Data Quality Matters 
Metadata is data that describes data! The enterprise Logical Data Model is a Metadata Model because it describes the elements and structure of the data required to support the core activities of the enterprise – Business Functions.
It is a mistake to look to processes for data. Processes do not create data! Business Functions do. A Process simply defines the order in which Business Functions are carried out in response to a specific trigger in order to arrive at a predefined outcome.
It is the steps in the Process – the Business Functions – that create, transform and use data. Know your Business Functions and you will know your data.
It is also a mistake to talk about ‘data quality rules’. These are Business Rules that are an embedded part of a Business Function. These should always be defined at the logical, rather that the physical, i.e. they should refer to entities, attributes and relationships, rather than tables and and records. The implementation of these business rules would result in quality data being created.
Regards
John
Hi John
Thank you for your comments.
Of course the definition of metadata is “data bout data”. This is precisely the problem – it is vague and all encompassing which means, frequently, that multiple people talking about this can mean different things creating massive confusion.
Business processes map and relate business functions – therefore they are a perfectly reasonable place to start!
Business rules are once again all encompassing – not all business rules impact on data quality. Data quality rules are an accepted subset of business rules that measure the compliance of data to the business need. Once again. in the interests of accuracy I believe that the more precise we are about what we mean the less chance we have of causing confusion.
That being said, there are many more examples of metadata that can be listed – my list was by no means definitive.
Interesting perspectives but we can’t look at metadata in isolation. In practise metadata only adds value when it describes the underlying data. The proof of the pudding is in the eating. I too see a three tier model but would describe the tiers slightly differently.
Tier 1.
Conceptual data model, a high level 50,000 foot view of the enterprise. A bit like a map of the world, useful to orientate an diverse group of people or to check you’ve arrived at the right planet but useless for finding your way home.
Tier 2.
Logical data model. Third normal form data model using abstract concepts like party (customer, supplier, person or organisation, employee, employer etc). Excellent way of ensuring you only store data once, dependent on the key, the whole key and nothing but the key. Also a great base for ensuring the organisation can have a single version of the true but, to business folks a third normal form data model is about as interesting as a wiring diagram of their office. Do they care?No, not till the lights go off and then they will call an electrician, well what would you do? Business people have other objectives, they won’t care what goes on behind the light switch and do not want lessons in reading wiring diagrams thank you very much. What they want are devices that work when you switch them on, computers that give them the right answer, not a lesson in semantics.
Tier 3.
This brings us to the Access layer sometimes call the semantic layer. Built on top of the third normal form core the access layer presents the data the way the business likes to see it. For example sales people see a subset of that Party entity as “customer”, whereas to purchasing another subset of Party is “Vendor”, or “Supplier” depending on their preference. The KPIs can be predefined, by the appropriate business department, and exposed here to. The access layer is the interface between the abstract third normal form concepts and the real, tangible, objects that business people deal with every day.
Now the challenge becomes one of understanding how these tiers map together but that’s a lesser challenge than trying to build a tower of Babel.
Andy.