Executive Summary

In the Big Data era, the vision of virtually every large enterprise is to maximize the use of their information assets to build competitive advantage. Much of the focus to date has been around the use of storage technologies and analytical tools to accomplish this goal. However, a frequently overlooked piece of the puzzle is leveraging new methods for managing the data that connects the storage systems with downstream uses such as analytics. Without complete and clean data, the analytics become incomplete, inaccurate, and even misleading. This report focuses on the importance of creating golden, master records of critical organizational entities (e.g., customers, suppliers, and products) and why leveraging machine learning to make the process more agile is critical to success.

Master records are the fuel for organizational analytics; they represent a complete view of unique entities across the distributed, messy data environments of large organizations. Analytic tools rely on such records to ensure the data being pulled is relevant to the entity being analyzed and that all of the data is captured, ultimately ensuring completeness and trust in the result. The traditional methods for building these master records, like master data management (MDM) software platforms, have been effective at a small scale, but are struggling to keep pace in the current environment. Typical MDM tools rely heavily on manual programming of rules that match and merge records to create a single golden record. When a data environment grows too large and too diverse, however, this becomes an unscalable practice. Unsustainable amounts of time and expense are required to keep pace with the amount of data being captured and, most often, the initiative will not deliver the return on investment that is needed.

Now is the time when organizations need to evaluate a new approach to mastering their data, an approach that cost-effectively delivers these golden records at speed and scale, across domains, and with the ability to classify them so organizations can fully realize the benefits of their analytic endeavors. This approach is called agile data mastering, and it revolves around the use of human-guided machine learning to match, merge, and classify core organizational entities like customers, suppliers, and products. Machine learning algorithms employ probabilistic models that attempt to master raw data records while an internal expert validates the results, which tunes the algorithms, delivering the aforementioned benefits while also ensuring an underlying accuracy and trust in the results. This report dives deeper into the elements of agile data mastering and the methods that power it so companies across any industry and with any type of data environment can manage their data to support their digital transformation goals and maintain their relevance.