This post was first published by our partner, Silwood Technology, on their blog, and is republished with permission
Can your enterprise data catalog or metadata management system truly be called ‘enterprise’ if it cannot readily accommodate metadata from your enterprise application systems?
Have you ever wondered how to go about provisioning your enterprise data catalog or metadata management solution with information from your enterprise applications such as those from SAP, Oracle, Salesforce or Microsoft?
It may seem that this should be a simple and straightforward process as it is with many other data sources. Your vendors may even suggest that their software solution can achieve this easily. Or, that it will be a task their consultants and your technical specialists can accomplish quickly and without difficulty.
In practise, your vendor may not be able to do this withour extensive, high-risk work. Safyr is the only solution designed to quickly and easily extract useful, business-friendly metadata from your ERP and CRM applications
We have technical and / or commercial partnerships with many of the leading enterprise data catalog and metadata management vendors. This means that their customers can use Safyr to provision their solutions with ERP and CRM metadata from their systems, as implemented, quickly and accurately.
The realities of ERP and CRM metadata for your data catalog
The reality is that this is not a simple challenge to overcome without specialist capabilities. If you have already tried to do this you may have realised how complex, time consuming and costly this can be to complete. It may even delay or derail your project.
There are some initial questions to answer when you consider provisioning your solution with metadata from these types of systems.
For example, do you really need to import all the tables and attributes from those applications? Whilst this is certainly possible, do you want tables which contain no data or are intermediate tables and contain no transactional data? A typical SAP ECC or S/4HANA system contains about 100,000 tables and 1 million attributes. Is it all appropriate for your data catalog?
Or, do you want to be able to select those tables which represent specific business functions or groupings? For example, Bill of Materials, Product Master, Customers, Sales, General Ledger etc. This might be more useful for your business users than having to guess which tables they need, or wade through a lot of results from searches. How will you be able to create the relevant subsets required for this?
How will the solution identify and synchronise with changes to your source metadata? This changes regularly in some applications, for example SAP BW or Salesforce. How will they be identified and incorporated?
These questions are more difficult to answer because the useful metadata, by which I mean usable table and attribute descriptions rather than physical names, relationships and other information, is not easy to find. It is not in the application’s database system catalog so a database scanner or connector is of no value.
In addition, the quantity and complexity of the metadata involved means that it is almost impossible to create business relevant subsets unless you have access to specialist metadata curation software.
Finally, it is also important to know how you can populate your enterprise data catalog with data from your application systems and keep it synchronised with changes to the sources?
Solving three main challenges
If you have, or are considering, or implementing an enterprise metadata management or data catalog from one of our partner solutions you can explore how our product Safyr allows you to successfully curate and ingest metadata from your SAP, Oracle, Salesforce or Microsoft applications.
ERP and CRM packages from vendors such as SAP, Salesforce, Oracle and Microsoft present you with 3 main challenges before you can use their metadata with your data management projects. If you cannot overcome these significant hurdles quickly the results can be delays, cost overruns, under-delivery and in some extreme cases project cancellation.
- Challenge 1: Finding and accessing your metadata. The ‘business friendly’ metadata that you need from these systems is not in their standard relational database system catalog. This means that using standard scanners will not give you anything useful from which to work to find what you need. The metadata is held in a series of data dictionary tables or hidden behind an API so it can be difficult to get at. It is also vital that you are able to find all the customisations you have made to your implementation of the package.
- Challenge 2: Curating your metadata. Once you have accessed the metadata, you need to be able to organise it to identify relationships between tables and other objects. This is essential so that you can easily curate the metadata into relevant subsets. The problem is that these systems have very large and data models which makes navigation difficult. For example a typical SAP system has over 90,000 tables and even a relatively modest JD Edwards application has over 5,000.
- Using your metadata with other products and technologies. Once you have identified the metadata you need for your project you will want to use it to provision other environments. These could be data catalog or governance platforms, enterprise metadata management, data warehouse, ETL or data modeling tools.
We developed Safyr® to enable you to dramatically shorten the time to value for projects which involve data from the main ERP and CRM packages by giving you the means to solve these challenges quickly and cost-effectively..
Safyr reduces the time, cost and resources needed for ERP metadata discovery, often by as much as 90%, whilst at the same time improving accuracy and compliance.