Metadata can be defined as data describing other data. It varies from simple catalog information, such as tables in a database, their names, sizes, columns, type of data, and number of rows, to more complex information, such as purpose of the data, who uses it and why, who owns it, who created it and when, who is allowed to change it, and so on.
It is very common to see different LOBs have different and sometimes conflicting definitions of apparently simple data elements. The more cross-functional data integration there is, the higher the possibility of misinterpretations and disparate understanding regarding data elements. Metadata management becomes more important as repositories are integrated and shared among multiple LOBs within the company.
Before a Customer MDM type of data integration, the many LOBs operate in silos. This often generates disagreements, but a common understanding and a unified jargon typically prevails. Moreover, the combination of many years of operation and a somewhat more specialized type of communication lowers the chances of misinterpretations and inconsistencies.
However, there are differences among the multiple siloed systems within the enterprise. Besides, a new Customer MDM repository will generally have its own nomenclature as well. As an example, a customer in a legacy CRM system may be either a prospect or a current customer, while the concept of prospect may not exist in the legacy AR system at all. When both CRM and AR are migrated to a customer hub, a prospect may be a party only, while an existing customer is the combination of a party and an account. Further classification of party and account types can complicate understanding and usage even more.
The previous example is just the tip of the iceberg. Many business processes and associated data elements will be impacted by a Customer MDM program, making it extremely important to properly document the many aspects of the data and its usage.
A particular type of metadata is data lineage. Data lineage handles the tracking of data: where it comes from, where it goes to, and how it is transformed. It is a particularly important component of data migration since understanding the source of the data and applied transformations can be critical when interpreting information in the new repository.
Companies typically underestimate the importance of metadata management in general, and more specifically during data migration projects. Metadata management is normally postponed when data migration projects are behind schedule because it doesn't necessarily provide immediate benefit. However, in the long run, it becomes critical. It is common to see data issues later, and without proper metadata or data lineage it becomes difficult to assess the root cause of the problem.
Metadata itself is data, thus it is commonly maintained in a data repository. There are specific metadata management products that may or may not be already included with an MDM solution. Sometimes those products can be cumbersome to use or take a long time to set up. Companies can implement their own metadata repository using a normal set of database tables, but that can take even longer. As usual, a diligent analysis should be performed before deciding on the best alternative.
Metadata management projects have a relatively high rate of failure. It happens usually because companies don't correctly define scope, set the correct expectations, and/or engage the right people. Table 3.1 presents some common problems faced by companies along with comments and suggestions for mitigating the issues.
Table 3.1 Examples of Metadata Management Problems.
| Problem | Comments and Mitigation |
| Not providing the foundation for or encouraging metadata management as a routine activity. | Metadata is a type of documentation. As such, people approach it as a hurdle. Companies need to provide a viable infrastructure for metadata management to facilitate the practice. Furthermore, it needs to communicate and demonstrate its need as an incentive to make it part of the culture. |
| Using people familiar with the metadata repository to document the information instead of people familiar with the actual data and business rules. | A metadata repository or whatever else is used to create metadata is just a tool. Familiarity with it is obviously important, but no more than the knowledge about the data. It is a lot easier to train somebody familiar with the data to learn the tool than the other way around. |
| Expecting that one person or one particular LOB can do the job alone. | A Customer MDM repository is likely the integration of many data sources, and it is used by many LOBs. Therefore, it needs to be documented from all facets. This exercise in itself can be the foundation for creating a common understanding of data elements, their business purposes, and reconciling any differences. |
| Expecting to go from 0 to 100 within a single massive metadata management project. | Everything must have a business purpose, and metadata is no different. Simply creating metadata for the sake of it can lead to frustration, skepticism, and consequent failure. It is important to establish a goal, and prioritize what needs to be documented. Requiring every single element to be documented may be ideal, but not viable. |
Most companies are behind with their metadata creation, and as such, will likely require a dedicated project to catch up. But do so wisely. Understand the business needs and how the metadata repository can support the existing business processes, data governance, and data stewardship. Don't bite off more than you can chew.
For a metadata management program to be successful, it needs to be accessible to everybody that needs it, either from a creation or a consumption perspective. It should also be readily available to be used as a byproduct of other activities, such as data migration and data cleansing. Remember, metadata is documentation, and the closer it is generated to the activity affecting it, the better.