CHAPTER 24
The Value of Project Management: A Measurement System

JAMES S. PENNYPACKER, CEO, DANCE COMMUNICATIONS, LLC

More than ever, investment in initiatives designed to improve organizational performance must be justified. Whether it’s the implementation of a project management methodology, a project office, project management software, or project management training, these initiatives must deliver positive and tangible results. The good news is that tangible measures of project management value and performance can be established by asking the right questions and developing an appropriate measurement system.

Over the past few years, major companies from a variety of industries—information technology, manufacturing, pharmaceutical, new product development, government, and professional services—have initiated projects to create measurement programs to measure the value that project management provides to their organizations. The goals of these project management measurement programs were to:

• Provide tangible metrics to senior management, on the value of implementing systematic project management methods in order to reinforce the business case for project management improvement across the organization.

• Boost customer and project team morale by sharing statistics that show the value their work adds to the organization and the improvement they can achieve.

• Track on-going project management performance and the business impact of project management to the organization.

• Initiate metric-based efforts to help streamline the project portfolio.

Project management value measurement programs (PM Value¹ Initiatives) consist of three-phase, six-step programs designed to bring a measurement team from an introduction to project management-focused measurements through design, development, and implementation of a project management value measurement program (see Figure 24-1).

PM VALUE INITIATIVE: PHASE ONE

Phase One focuses on educating a measurement team on the PM Value Measurement Program to help them understand and enable them to clearly identify the program’s objectives and goals. Organizational constructs that affect the PM Value Measurement Program identified, including stakeholders, organizational mission and strategies, organizational structure, key business processes, project management maturity, prior project management improvement initiatives, current measurement systems, and data availability.

Step 1: Measurement Readiness Planning

Activities in this step ensure that the measurement initiative is clearly understood and aligned to support the organization’s strategies—an essential element to support sustained success of the initiative. The primary project management, business unit, and organization goals that influence the development of the measures of project management usually value include:

1. Organizational Goals and Objectives

• Reduced costs.

• Improved quality.

• Improved timing.

• Improved productivity.

2. Project Management Goals and Objectives

• More predictable project performance.

• More repeatable project performance.

• Improved ability to execute projects.

FIGURE 24-1. THE PM VALUE MEASUREMENT PROGRAM INITIATIVE PROCESS

• More effective resource management.

• Improved internal and external customer satisfaction.

• Better alignment of projects to business strategy.

• More effective risk management.

• Reduced learning curve for new project managers.

3. PM Value Initiative Goals and Objectives

• Measure the business impact of project management improvement initiatives.

• Compare the costs to benefits of project management improvement initiatives.

• Determine if a project management improvement initiative is accomplishing its objectives.

• Identify the strengths and weaknesses in project management processes.

• Establish a database of key project management measures.

• Understand the infrastructure required to support a measurement program.

• Gain a sense of whether the project portfolio is as productive as possible.

• Spread the acceptance of project management throughout the enterprise

• Help attain project management and organizational goals and objectives.

This step educates the measurement team on the issues of project management value measurement and better prepares them to make key decisions throughout the initiative concerning the program’s objectives and measures and the implementation approach.

PM VALUE INITIATIVE: PHASE TWO

Phase Two efforts plan the initiative and engage the team to identify measures, develop a PM Value Scorecard, and plan the implementation of the measurement program. After putting a PM Value Initiative Plan and Schedule in place, subsequent steps in this phase continue to build on the team’s understanding of the project management measurement program and engage the team to develop the PM Value Scorecard and PM Value Implementation Plan.

Step 2: Initiative Planning

In this step the team aligns around the measurement program’s objectives, scope, development approach, timeline, deliverables, and implementation strategy. It collaboratively develops a PM Value Initiative Plan and PM Value Initiative Schedule (see Figure 24-2).

Step 3: Measures Development

In the Measures Development step, the team creates and prioritizes the initial list of measures for the Scorecard. It is the initial pass at identifying and prioritizing measures with the primary activity in the step being a collaborative development workshop. A comprehensive list of measures developed keeping in mind that they need to be logically linked to the goals described above. The measures also need to meet the criteria for good measures, which means that the measures selected:

• Provide meaningful information.

• Are supported by valid data that is cost effective to capture.

• Are acceptable by stakeholders.

• Are repeatable.

FIGURE 24- 2. HIGH-LEVEL SCHEDULE OF PM VALUE INITIATIVE ACTIVITIES

• Are actionable.

• Align to organizational objectives.

A sample list of prospective measures developed by the measurement team is shown in Figure 24-3.

COST MEASURES

Project cost

ROI

Product cost variance to plan

Start-up costs

Efficiency of delivery

Project profitability

Product unit cost

Start-up cost variance to plan

Resource utilization

Market share

Cost of capital

PRODUCTIVITY MEASURES

Project milestone performance

Project success rate

Avg. sales per development FTE

Process improvement

Alternatives assessment

Downtime

Capacity/resource planning

QUALITY MEASURES

Requirements performance

Customer satisfaction

Lessons learned implemented

Project status communication

# scope changes/phase

Effectiveness

AARs

Rework

Internal customer satisfaction

Leadership capability

Staffing conformance to plan

Project risk management

PM training satisfaction

TIMING MEASURES

Predictability of delivery

Time to market

Project cycle time

Successful phase exits

Project planning

FIGURE 24-3. PARTIAL LIST OF PROSPECTIVE MEASURES OF PROJECT MANAGEMENT VALUE

The measurement team then prioritizes and selects measures to comprise the PM Value Scorecard. A simple prioritization process can be used: develop criteria for ranking the list of measures in order of importance on a scale of 1–5, and have each of the measurement team members rank the list; calculate average rankings and develop a prioritized list was developed for review in Step 4 (see Figure 24-4).

Step 4: Scorecard Development

In this step the team reviews the prioritized measures information developed to date and develops measure packages (see below) and a cohesive PM Value Scorecard. The team first engages in measures review, prioritization validation, and measure package definition. That information is then used to construct the Scorecard for review and acceptance by the measurement team in preparation for implementation.

A comprehensive definition of each measure is included in a measure package to support the initial implementation and ongoing collection of data. Each measure package includes the following elements:

• Measure (What): The data to be collected must be clearly identified.

FIGURE 24-4. SAMPLE LIST OF PRIORITIZED MEASURES

• Objective (Why): The measure’s objective must be clearly defined. Why is it being collected? How will it be interpreted? What will it tell us? The measurement team must understand the objective of each measure.

• Data Capture (How): The mechanism for collecting the data must be identified.

• Timing (When): The timing of data collection must be defined. Data collection must be properly timed to match the type of data and objective. PM value measures are not intended to track individual project progress, so there would most likely not be a need to collect data monthly. Typically a quarterly or longer interval will support the objectives of the initiative.

• Data Location (Where): The location of the data must be identified.

• Data Contact (Who): The person responsible for maintaining the data must be identified. Who will provide the data? What is the reliability of this source?

Information from the measure packages is used to create a PM Value Scorecard, which is a collection and reporting tool for keeping score and reporting progress (see Figure 24-5).

FIGURE 24-5. SAMPLE PM VALUE SCORECARD

Step 5: Measurement Program Implementation Planning

The implementation planning efforts in Step 5 define the framework around measurement processes and data collection that will be used to support ongoing measures program implementation. Key activities in this step include development of an implementation strategy and process.

FIGURE 24-6. THE PM VALUE MEASUREMENT PROGRAM PROCESS

The PM Value Measurement Program process shown below (see 24-6) describes a systematic approach to project management performance improvement through an ongoing process of establishing project management measures, collecting, analyzing, reviewing, and reporting performance data; using that data to drive performance improvement; and using lessons learned to continuously improve the PM Value Measurement Program process.

PM VALUE INITIATIVE: PHASE THREE

Phase Three includes an initial implementation of the program and the transition to ongoing execution of the program.

Step 6: Measurement Program Implementation

The PM Value Measurement Program Implementation is an ongoing effort to execute the program as documented in the Implementation Plan, using the Measures Packages to reinforce the data requirements, collection timing, and data contact responsibilities. Step 6 begins with the preparation for the initial collection-analysis-reporting cycle and continues through transition of ongoing program execution and continuous improvement responsibilities.

LESSONS LEARNED

Organizational strategies and objectives set the foundation for effective measurement programs. It’s essential to understand how the critical elements of the organization’s strategies and objectives are linked to the measures that comprise the PM Value Scorecard.

You need to have a very clear idea of the measurement program stakeholders and what their needs and expectations are regarding the program (there are often huge differences in expectation among stakeholders—setting those expectations through clear communication of program goals is a key to success).

Clearly identify measurement program goals and objectives. Without this clarity, selecting the right set of critical few measures will be difficult.

In most best-in-class organizations, measurement initiatives are introduced and continually championed and promoted by top executives. When measurement initiatives are introduced from the bottom up, getting senior management buy-in is crucial and may take significant effort. Be prepared to make that effort.

Develop a clear understanding of measurement terminology, which tends to be confusing and inconsistent, but needs to be understood and agreed upon by the measurement team and program stakeholders.

Communication is crucial for establishing and maintaining a successful measurement program. It should be multidirectional, running top-down, bottom-up, and horizontally within and across the organization.

The driving force to create a new or improved measurement program is usually a threat to the organization (often a crisis or strong competition). For organizations that are strategically developing measurement programs to enhance their competitive advantage, rather than reacting to their business environment, a sense of urgency must be nurtured and driven by individuals who understand the value of measurement and can evangelize the need for developing a measurement culture. Again, this takes enormous effort and communication.

It’s critical, and very difficult, to limit the number of measures in the Scorecard. Selecting those critical few measures sharpens the stakeholders’ understanding of the issues. Too many measures confuse and complicate (the measurement team can’t try to please everyone—selecting too many measures will ultimately kill the program).

Pilot the measurement program before full implementation. And implementation should come in phases—implement a critical few high-value measures at first and identify more detailed measures when the organization has developed a measurement culture and is ready to collect and analyze more complex measures.

Successful deployment of a measurement program requires a successful system of accountability—that is, all stakeholders need to buy into measurement by assuming responsibility for some part of the measurement process (sponsorship, analysis, data collection and monitoring, evangelism, etc.).

Benchmark against industry standards if possible.

Identify a central area of responsibility for the measurement program.

You need to determine what counts as a project (what exactly will be measured).

Reinforce the fact that PM Value measurement is measuring performance change due to project management. Measures, therefore, are process-focused, not project-focused (you are not trying to measure the progress of a particular project).

The measures selected are highly influenced by the project management maturity of the organization. Level one organizations generally need to focus on process compliance and simple cost and/or schedule measures. As the organization matures in their project management capability, more sophisticated measures can be used.

Analysis is one of the most important steps in PM Value measurement, yet it is often one that is neglected. The insight gained from effective analysis (particularly determining root causes of the results measured) is what makes measurement a valuable business tool.

Feedback is one of the best assets for continuous improvement. Seek it and use it.

REFERENCES

¹ The PM Value system is a part of the performance measurement practice of Project Management Solutions, Inc. (www.pmsolutions.com). The material in this chapter was originally presented on behalf of PM Solutions at the 2002 PMI Global Congress.

FURTHER READING

Crawford, Deborah B. Mastering Performance Measurement. White paper, PM Solutions, 2009.

Oswald, J., & Pennypacker, J. S. The Value of Project Management: The Business Case for Implementation of Project Management Initiatives. Proceedings of the Project Management Institute Annual Seminars & Symposium, 2002.

Pennypacker, J. S. The Value of Project Management: A Center for Business Practices Research Report. Havertown, PA: Center for Business Practices, 2001.

Pennypacker, J. S. (Ed.). Justifying the Value of Project Management. Havertown, PA: Center for Business Practices, 2003.