Unraveling the Metadata Conundrum: Understanding the Disconnect in IT Projects

In any IT project, one of the most dangerous disconnects is often the one we fail to realize exists. It stems from the assumption that using the same terms implies we have the same understanding of their meaning. However, in many cases, business terms are only loosely understood or, at worst, have entirely different meanings depending on the business context. In this post, we explore the concept of metadata management and how this can help to reduce ambiguity and bridge this disconnect.

But first, a riddle. What is brown and sticky?

Ambiguous Business Terms

Consider the example of a Recourse Business Unit used by a client.

While the term is commonly used throughout the organization to refer to the business unit responsible for KYC (Know Your Customer) processes, the credit risk team interprets it as the unit that carries the credit risk.

This discrepancy becomes particularly problematic when implementing enterprise-wide projects like master data management or ERP systems, or when attempting to reuse tactical solutions to reduce overall IT expenditure.

Is Metadata Management the Solution?

To address this issue, many organizations have initiated metadata management initiatives. However, even the term “metadata” itself can be ambiguous and lead to confusion.

It is crucial to establish a clear and shared understanding of critical business metadata.

This includes developing a business terms glossary to define business terms like Contracts or Profit, ensuring consistent interpretation across departments. Additionally, maintaining a data dictionary for essential attributes, along with their use, data lineage, ownership, and data quality rules, becomes vital.

Understanding the source, or provenance, of data is particularly important to avoid inconsistencies in key reports or metrics resulting from multiple versions of the truth.

Data governance is the key to useful metadata

Incorporating data governance or data quality rules, such as mandating valid banking details for supplier records, helps evaluate data fitness for purpose, identifies broken business processes, and enables root cause analysis. Technical metadata, including data flows and data models, also play a role in understanding the overall data landscape.

Focus on key processes

When tackling data documentation, organizations should focus on key business processes to identify the most critical data elements requiring attention. Attempting to address everything at once can quickly become overwhelming. By narrowing the scope to a manageable number of key indicators, data attributes, and systems, the organization can effectively manage the complexity.

Active metadata

Lastly, organizations should be cautious when investing in passive metadata documentation that quickly becomes outdated and offers limited reusability. Instead, metadata should be actively utilized, such as by implementing data quality rules within a data quality platform. This approach allows for real-time monitoring, adaptability to changes, and actionable insights to improve data quality.

Conclusion

So, what is brown and sticky?

Well, the answer is a stick.

While it may seem unrelated, the riddle serves as a reminder that assumptions can lead to misconceptions. By focusing on establishing shared understanding and clear metadata definitions, organizations can overcome the metadata conundrum and ensure data quality and consistency across their projects and systems.

Responses to “Unraveling the Metadata Conundrum: Understanding the Disconnect in IT Projects”

http://integrated-modeling-method.com

March 28

Metadata is data that describes data! The enterprise Logical Data Model is a Metadata Model because it describes the elements and structure of the data required to support the core activities of the enterprise – Business Functions.

It is a mistake to look to processes for data. Processes do not create data! Business Functions do. A Process simply defines the order in which Business Functions are carried out in response to a specific trigger in order to arrive at a predefined outcome.

It is the steps in the Process – the Business Functions – that create, transform and use data. Know your Business Functions and you will know your data.

It is also a mistake to talk about ‘data quality rules’. These are Business Rules that are an embedded part of a Business Function. These should always be defined at the logical, rather that the physical, i.e. they should refer to entities, attributes and relationships, rather than tables and and records. The implementation of these business rules would result in quality data being created.

Regards
John

Reply
1. garymdm
  
  March 28
  
  Hi John
  
  Thank you for your comments.
  
  Of course the definition of metadata is “data bout data”. This is precisely the problem – it is vague and all encompassing which means, frequently, that multiple people talking about this can mean different things creating massive confusion.
  
  Business processes map and relate business functions – therefore they are a perfectly reasonable place to start!
  
  Business rules are once again all encompassing – not all business rules impact on data quality. Data quality rules are an accepted subset of business rules that measure the compliance of data to the business need. Once again. in the interests of accuracy I believe that the more precise we are about what we mean the less chance we have of causing confusion.
  
  That being said, there are many more examples of metadata that can be listed – my list was by no means definitive.
  
  Reply
Andy Bury

March 30

Interesting perspectives but we can’t look at metadata in isolation. In practise metadata only adds value when it describes the underlying data. The proof of the pudding is in the eating. I too see a three tier model but would describe the tiers slightly differently.
Tier 1.
Conceptual data model, a high level 50,000 foot view of the enterprise. A bit like a map of the world, useful to orientate an diverse group of people or to check you’ve arrived at the right planet but useless for finding your way home.
Tier 2.
Logical data model. Third normal form data model using abstract concepts like party (customer, supplier, person or organisation, employee, employer etc). Excellent way of ensuring you only store data once, dependent on the key, the whole key and nothing but the key. Also a great base for ensuring the organisation can have a single version of the true but, to business folks a third normal form data model is about as interesting as a wiring diagram of their office. Do they care?No, not till the lights go off and then they will call an electrician, well what would you do? Business people have other objectives, they won’t care what goes on behind the light switch and do not want lessons in reading wiring diagrams thank you very much. What they want are devices that work when you switch them on, computers that give them the right answer, not a lesson in semantics.
Tier 3.
This brings us to the Access layer sometimes call the semantic layer. Built on top of the third normal form core the access layer presents the data the way the business likes to see it. For example sales people see a subset of that Party entity as “customer”, whereas to purchasing another subset of Party is “Vendor”, or “Supplier” depending on their preference. The KPIs can be predefined, by the appropriate business department, and exposed here to. The access layer is the interface between the abstract third normal form concepts and the real, tangible, objects that business people deal with every day.
Now the challenge becomes one of understanding how these tiers map together but that’s a lesser challenge than trying to build a tower of Babel.

Andy.

Reply

Discover more from Data Quality Matters

Subscribe now to keep reading and get our new posts in your email.