Chapter 1 presented the concept of data management within the overall concept of the enterprise and information technology. Chapter 2 provides a detailed overview of data management that includes:
Chapters 3 through 12 explore each of the ten data management functions and their component activities in more detail. Each chapter begins with an introduction that includes that function’s context diagram. The rest of each chapter explains key concepts, and the activities in the diagram in depth. The last part of each chapter includes some guiding principles, organizational and cultural discussions, followed by a bibliography.
Finally, Chapter 13 covers topics related to professional development for data management professionals. All of these chapters together provide a basic body of knowledge regarding the data management profession, and data management functions and activities.
This chapter will cover process, people, and technology as it relates to overall data management. Chapters 3 through 12 concentrate on the process of each data management function.
2.1 Introduction
Data management is a function that is also known as a high-level business process. It consists of:
Data management may also be the name of a program, which is an on-going initiative that includes several related projects. The term “data management program” can be substituted for “data management function”. The major elements of data management are summarized in the context diagram shown in Figure 2.1.
Figure 2.1 Data Management Context Diagram
2.2 Mission and Goals
The mission of the data management function is to meet and exceed the information needs of all the stakeholders in the enterprise in terms of information availability, security, and quality.
The strategic goals of the data management function are:
Other non-strategic goals of data management include:
While the goals of data management are constant and consistent across enterprises, the objectives for data management at any enterprise vary from year to year. Objectives should be “SMART”–specific, measurable, achievable (or actionable), realistic, and timely, with a specified target timeframe.
2.3 Guiding Principles
Overall and general data management principles include:
2.4 Functions and Activities
The process of data management is captured in functions and activities. The ten component functions of data management are:
Many data management activities overlap in scope with other recognized functions, within and outside IT. The DAMA-DMBOK Guide does not attempt to identify which processes are exclusive to a data management function. The only objective is to describe the full scope and context of data management.
Many data management activities described here are not performed in every enterprise. In fact, few organizations have plans, policies, and programs in each of the ten functions. In a given enterprise, certain functions will be more relevant, at least at any one point in time, and will receive higher priority than other functions. The enterprise will rightly invest more attention, time, and effort in some functions and less in others.
How each enterprise implements these activities varies widely. Each organization must determine an implementation approach consistent with its size, goals, resources, and complexity. However, the essential nature and the fundamental principles of data management remain the same across the spectrum of enterprises.
2.4.1 Data Management Activities
Each of these functions decomposes into activities. In a few cases, the activities further decompose into sub-activities. While noun phrases name functions, verb phrases name activities and sub-activities.
2.4.2 Activity Groups
Each activity belongs to one of four Activity Groups:
Each data management activity fits into one or more data management activity groups, as shown in Table 2.1.
Functions |
Planning Activities (P) |
Control Activities (C) |
Development Activities (D) |
Operational Activities (O) |
|
1. Data Governance |
1.1 Data Management Planning |
1.2 Data Management Control |
|||
2. Data Architecture Management |
2. Data Architecture Management (all) |
||||
3. Data Development |
3.3 Data Model and Design Quality Management |
3.3 Data Model and Design Quality Management |
3.1 Data Modeling, Analysis, and Solution Design 3.2 Detailed Data Design 3.4 Data Implementation |
||
4. Data Operations Management |
4.1 Database Support 4.2 Data Technology Management |
4.1 Database Support 4.2 Data Technology Management |
4.1 Database Support 4.2 Data Technology Management |
||
5. Data Security Management |
5.1 Understand Data Security Needs and Regulatory Requirements 5.2 Define Data Security Policy 5.3 Define Data Security Standards |
5.5 Manage Users, Passwords, and Group Membership 5.6 Manage Data Access Views and Permissions 5.7 Monitor User Authentication and Access Behavior 5.8 Classify Information Confidentiality 5.9 Audit Data Security |
5.4 Define Data Security Controls and Procedures |
||
6. Reference and Master Data Management |
6.1 Understand Reference and Master Data Integration Needs 6.2 Understand Reference and Master Data Sources and Contributors 6.3 Define the Data Integration Architecture |
6.5 Define and Maintain Match Rules 6.6 Establish “Golden” Records 6.7 Define and Maintain Hierarchies and Affiliations |
6.4 Implement Reference and Master Data Management Solutions 6.8 Plan and Implement Integration of New Data Sources 6.10 Manage Changes to Reference and Master Data |
6.9 Replicate and Distribute Reference and Master Data |
|
7. Data Warehousing and Business Intelligence Management |
7.1 Understand Business Intelligence Information Needs 7.2 Define and Maintain the DW / BI Architecture |
7.6 Monitor and Tune Data Warehousing Processes 7.7 Monitor Business Intelligence Activity and Performance |
7.3 Implement Data Warehouses and Data Marts 7.4 Implement BI Tools and User Interfaces |
7.5 Process Data for Business Intelligence |
|
8. Document and Content Management |
8.1 Documents / Records Management 8.2 Content Management |
8.1 Documents / Records Management 8.2 Content Management |
8.1 Documents / Records Management 8.2 Content Management |
||
9. Meta-data Management |
9.1 Understand Meta-data Requirements 9.2 Define the Meta-data Architecture 9.3 Develop and Maintain Meta-data Standards |
9.6 Integrate Meta-data 9.7 Manage Meta-data Repositories 9.8. Deliver and Distribute Meta-data |
9.4 Implement a Managed Meta-data Environment |
9.5 Create and Maintain Meta-data 9.9 Query, Report, and Analyze Meta-data |
|
10. Data Quality Management |
10.4 Define Data Quality Metrics 10.5 Define Data Quality Business Rules 10.7 Set and Evaluate Data Quality Service Levels |
10.8 Continuously Measure and Monitor Data Quality 10.9 Manage Data Quality Issues 10.12 Monitor Operational DQM Procedures and Performance |
10.2 Define Data Quality Requirements 10.3 Profile, Analyze, and Assess Data Quality 10.6 Test and Validate Data Quality Requirements 10.11 Design and Implement Operational DQM Procedures |
10.1 Develop and Promote Data Quality Awareness 10.10 Clean and Correct Data Quality Defects |
Table 2.1 Activities by Activity Groups
2.5 Context Diagram Overview
Each context diagram in this Guide contains a definition and a list of goals at the top of the diagram. In the center of each diagram is a blue box containing the list of that function’s activities, and in some cases, sub-activities. Each chapter describes these activities and sub-activities in depth.
Surrounding each center activity box are several lists. The lists on the left side (flowing into the activities) are the Inputs, Suppliers, and Participants. The list below the box is for Tools used by the Activities. The lists on the right side (flowing out of the activities) are Primary Deliverables, Consumers, and sometimes Metrics.
These lists contain items that apply to that list’s topic. By no means are they exhaustive, and some of the items will not apply to all organizations. These lists are meant as a context framework, and will grow over time as the data management profession grows and matures.
For convenience of comparison, all of the contents of each function list are included in appendices.
2.5.1 Suppliers
Suppliers are the entities responsible for supplying inputs for the activities. Several suppliers relate to multiple data management functions. Suppliers for data management in general include Executives, Data Creators, External Sources, and Regulatory Bodies. The suppliers for each data management function are listed in Appendix A1.
2.5.2 Inputs
Inputs are the tangible things that each function needs to initiate its activities. Several inputs are used by multiple functions. Inputs for data management in general include Business Strategy, Business Activity, IT Activity, and Data Issues. The inputs for each data management function are listed in Appendix A2.
2.5.3 Participants
Participants are involved in the data management process, although not necessarily directly or with accountability. Multiple participants may be involved in multiple functions. Participants in data management in general include Data Creators, Information Consumers, Data Stewards, Data Professionals, and Executives. The participants in each data management function are listed in Appendix A3.
2.5.4 Tools
Data management professionals use tools to perform activities in the functions. Several tools are used by multiple functions. Tools for data management in general include Data Modeling Tools, Database Management Systems, Data Integration and Quality Tools, Business Intelligence Tools, Document Management Tools, and Meta-data Repository Tools. The tools used by each data management function are listed in Appendix A4.
2.5.5 Primary Deliverables
Primary deliverables are the tangible things that each function is responsible for creating. Several primary deliverables are created by multiple functions. The primary deliverables for Data Management in general include Data Strategy, Data Architecture, Data Services, Databases, and Data, Information, Knowledge and Wisdom. Obviously, ten functions would have to cooperate to provide only eight deliverables. The primary deliverables of each data management function are listed in Appendix A5.
2.5.6 Consumers
Consumers are those that benefit from the primary deliverables created by the data management activities. Several consumers benefit from multiple functions. Consumers of data management deliverables in general include Clerical Workers, Knowledge Workers, Managers, Executives, and Customers. The consumers of each data management function are listed in Appendix A6.
2.5.7 Metrics
The metrics are the measurable things that each function is responsible for creating. Several metrics measure multiple functions, and some functions do not (in this edition) have defined metrics. Metrics for data management include Data Value Metrics, Data Quality Metrics, and Data Management Program Metrics. The metrics for each data management function are listed in Appendix A7.
2.6 Roles
The people part of data management involves organizations and roles. Many organizations and individuals are involved in data management. Each company has different needs and priorities. Therefore, each company has a different approach to organizations, and individual roles and responsibilities, for data management functions and activities. Provided here is an overview of some of the most common organizational categories and individual roles.
Suppliers, participants, and consumers, as mentioned in the context diagrams, may be involved in one or more data management organizations, and may play one or more individual roles. It would be beyond the scope of this work to identify and define all possible suppliers, participants, and consumers, and all the roles and organizations that would apply. However, it is possible to outline the high-level types of organizations and individual roles.
2.6.1 Types of Organizations
Table 2.2 includes descriptions of the most common types of data management organizations.
Types of Data Management Organizations |
Description |
Data Management Services organization(s) |
One or more units of data management professionals responsible for data management within the IT organization. A centralized organization is sometimes known as an Enterprise Information Management (EIM) Center of Excellence (COE). This team includes the DM Executive, other DM Managers, Data Architects, Data Analysts, Data Quality Analysts, Database Administrators, Data Security Administrators, Meta-data Specialists, Data Model Administrators, Data Warehouse Architects, Data Integration Architects, and Business Intelligence Analysts. May also include Database Administrators (DBA), although DBAs are found within both Software Development organizations and Infrastructure Management organizations. May also include Data Integration Developers and Analytics / Report Developers, although often they remain in Software Development organizations with other developers. |
Data Governance Council |
The primary and highest authority organization for data governance in an organization. Includes senior managers serving as executive data stewards, along with the DM Leader and the CIO. A business executive (Chief Data Steward) may formally chair the council, in partnership with the DM Executive and Data Stewardship Facilitators responsible for council participation, communication, meeting preparation, meeting agendas, issues, etc. |
Data Stewardship Steering Committee(s) |
One or more cross-functional groups of coordinating data stewards responsible for support and oversight of a particular data management initiative launched by the Data Governance Council, such as Enterprise Data Architecture, Master Data Management, or Meta-data Management. The Data Governance Council may delegate responsibilities to one or more Data Stewardship Committees. |
Data Stewardship Team(s) |
One or more temporary or permanent focused groups of business data stewards collaborating on data modeling, data definition, data quality requirement specification and data quality improvement, reference and master data management, and meta-data management, typically within an assigned subject area, led by a coordinating data steward in partnership with a data architect and a data stewardship facilitator. |
Data Governance Office (DGO) |
A staff organization in larger enterprises supporting the efforts of the Data Governance Council, Data Stewardship Steering Committees, and Data Stewardship Teams. The DGO may be within or outside of the IT organization. The DGO staff includes Data Stewardship Facilitators who enable stewardship activities performed by business data stewards. |
Table 2.2 Types of Data Management Organizations
2.6.2 Types of Individual Roles
Table 2.3 contains a summary of many individual roles that may participate in data management activities.
2.7 Technology
The Technology section identifies and defines the categories of technology related to data management. Technology is covered in each chapter where tools are specifically mentioned.
2.7.1 Software Product Classes
The metrics are the measurable things that each function is responsible for creating. Several Metrics measure multiple functions, and some functions do not (in this edition) have defined metrics. Metrics for data management include Data Value Metrics, Data Quality Metrics, and DM Program Metrics. The metrics for each data management function are listed in Appendix A7.
Types of Data Management Individual Roles |
Description |
Business Data Steward |
A knowledge worker and business leader recognized as a subject matter expert who is assigned accountability for the data specifications and data quality of specifically assigned business entities, subject areas or databases, who will:
|
Coordinating Data Steward |
A business data steward with additional responsibilities, who will:
|
Executive Data Steward |
A role held by a senior manager sitting on the Data Governance Council, who will:
|
Data Stewardship Facilitator |
A business analyst responsible for coordinating data governance and stewardship activities, who will:.
|
Data Management Executive |
The highest-level manager of Data Management Services organizations in an IT department.. The DM Executive reports to the CIO and is the manager most directly responsible for data management, including coordinating data governance and data stewardship activities, overseeing data management projects, and supervising data management professionals. May be a manager, director, AVP or VP. |
Data Architect |
A senior data analyst responsible for data architecture and data integration. |
Enterprise Data Architect |
The senior data architect responsible for developing, maintaining, and leveraging the enterprise data model. |
Data Warehouse Architect |
A data architect responsible for data warehouses, data marts, and associated data integration processes. |
Data Analyst / Data Modeler |
An IT professional responsible for capturing and modeling data requirements, data definitions, business rules, data quality requirements, and logical and physical data models. |
Data Model Administrator |
Responsible for data model version control and change control. |
Meta-data Specialist |
Responsible for integration, control, and delivery of meta-data, including administration of meta-data repositories. |
Data Quality Analyst |
Responsible for determining the fitness of data for use. |
Database Administrator |
Responsible for the design, implementation, and support of structured data assets. |
Data Security Administrator |
Responsible for ensuring controlled access to classified data. |
Data Integration Architect |
A senior data integration developer responsible for designing technology to integrate and improve the quality of enterprise data assets. |
Data Integration Specialist |
A software designer and developer responsible for implementing systems to integrate (replicate, extract, transform, load) data assets in batch or near real time. |
Business Intelligence Architect |
A senior business intelligence analyst responsible for the design of the business intelligence user environment. |
Business Intelligence Analyst / Administrator |
Responsible for supporting effective use of business intelligence data by business professionals. |
Business Intelligence Program Manager |
Coordinates BI requirements and initiatives across the corporation and integrates them into a cohesive prioritized program and roadmap. |
Analytics / Report Developer |
A software developer responsible for creating reporting and analytical application solutions. |
Business Process Analyst |
Responsible for understanding and optimizing business processes. |
Enterprise Process Architect |
Senior business process analyst responsible for overall quality of the enterprise process model and enterprise business model. |
Application Architect |
Senior developer responsible for integrating application systems. |
Technical Architect |
Senior technical engineer responsible for coordinating and integrating the IT infrastructure and the IT technology portfolio. |
Technical Engineer |
Senior technical analyst responsible for researching, implementing, administering, and supporting a portion of the information technology infrastructure. |
Help Desk Administrator |
Responsible for handling, tracking, and resolving issues related to use of information, information systems, or the IT infrastructure. |
IT Auditor |
An internal or external auditor of IT responsibilities, including data quality and / or data security. |
Chief Knowledge Officer (CKO) |
The executive with overall responsibility for knowledge management, including protection and control of intellectual property, enablement of professional development, collaboration, mentoring, and organizational learning. |
Collaborators |
Suppliers or consortium participants of an organization. These may engage in data sharing agreements. |
Data Brokers |
Suppliers of data and meta-data often by subscription for use in an organization. |
Government and Regulatory Bodies |
Data Management rules of engagement in the market are specified and enforced by various government and regulatory bodies. Privacy, confidential, proprietary data, and information are key areas. |
Knowledge Workers |
Business analyst consumers of data and information who add value to the data for the organization. |
Table 2.3 Types of Individual Roles
2.7.2 Specialized Hardware
While most data technology is software running on general purpose hardware, occasionally specialized hardware is used to support unique data management requirements. Types of specialized hardware include:
2.8 Recommended Reading
Adelman, Sid, Larissa Moss, and Majid Abai. Data Strategy. Addison-Wesley, 2005. ISBN 0-321-24099-5. 384 pages.
Boddie, John. The Information Asset: Rational DP Funding and Other Radical Notions. Prentice-Hall (Yourdon Press Computing Series), 1993. ISBN 0-134-57326-9. 174 pages.
Bryce, Milt and Tim Bryce. The IRM Revolution: Blueprint for the 21st Century. M. Bryce Associates Inc., 1988. ISBN 0-962-11890-7. 255 pages.
DAMA Chicago Chapter Standards Committee, editors. Guidelines to Implementing Data Resource Management, 4th Edition. Bellevue, WA: The Data Management Association (DAMA International), 2002. ISBN 0-9676674-1-0. 359 pages.
Durell, William R. Data Administration: A Practical Guide to Successful Data Management. New York: McGraw-Hill, 1985. ISBN 0-070-18391-0. 202 pages.
Horrocks, Brian and Judy Moss. Practical Data Administration. Prentice-Hall International, 1993. ISBN 0-13-689696-0.
Kent, William. Data and Reality: Basic Assumptions in Data Processing Reconsidered. Authorhouse, 2000. ISBN 1-585-00970-9. 276 pages.
Kerr, James M. The IRM Imperative. John Wiley & Sons, 1991. ISBN 0-471-52434-4.
Newton, Judith J. and Daniel Wahl, editors. Manual For Data Administration. Washington, DC: GPO, NIST Special Publications 500-208, Diane Publishing Co., 1993. ISBN 1-568-06362-8.
Purba, Sanjiv, editor. Data Management Handbook, 3rd Edition. Auerbach, 1999. ISBN 0-849-39832-0. 1048 pages.