Chapter 5

Configuration Status Accounting and Metrics

This chapter describes maintaining and reporting information about the CM items and related actions, such as pending changes. A project may use configuration status and accounting to disseminate information, validate and verify actions, and to serve as the feedback mechanism for the overall PCM process. This chapter includes the following major sections:

5.1 Information Repository

Chapter 2 provided repository examples used in PCM and CM, and Chapter 3 provided examples mentioned in the PMBOK® Guide–Third Edition. Information repositories are essential to PCM. An information repository is a place (database, library, or file system) where project information is deposited for controlled storage, access, version, and version control. A project may have one or many repositories. The PCM plan could describe the repositories and the processes for their use. The PCM plan could also show the relationships and any interfaces among these repositories, their processes, and the people who use them. Configuration status accounting (CSA) involves acquiring and entering information into the applicable repository. For inputting and later locating information, it has been found useful for the repository to be organized systematically. WBS, OBS, or document hierarchies are examples of ways to organize the repository’s structure.

Based on the information gathered, a series of metrics are developed for analysis. Configuration status accounting (CSA) provides the data necessary to validate if approved changes have been consistent with the project’s objectives and can be traced to the project scope and its evolution. This data repository includes various benchmark, baseline, and production level metrics, in conjunction with the revision level history for all hardware, software, and documentation.

5.2 Reporting

CSA and metrics reporting produces management information for the project. This information can be used to baseline a production level metric, control changes to this metric, and track any improvement the metric provides the project over time. Once a baseline production level metric has been established, periodic tracking reports can be produced on CM activities, project quality, and measured performance over time.

5.2.1 Status

Status in the PCM context requires that PCM has control over all CIs, can show an audit trail and the artifacts of all changes including all versions of the CIs, and show the status of each version. Status of a change to the CI should indicate its stage in the life cycle, such as draft, in review, approved, not effective, effective, retired, or obsolete. Status also indicates whether changes are pending to a CI. The level of detail and control is related to risk, regulations, and standards. For example, it is expected that more resources will be expended on maintaining strict controls and records about scope than on the minutes of weekly project team meetings.

Status accounting tracks the status of approved changes. As such, PCM status accounting serves as a control mechanism to inform the project manager about the results of an approved change. The status of each CI is determined in agreement with CCM processes. Coordination and communication are essential to ensure that the changes that are planned and the data to be tracked within a given change are identified well in advance. A collection plan, even if it is as simple as a checklist, helps to ensure the necessary data is captured. Necessary data may include which CIs are addressed in given technical and verification reviews, so that the required status information is provided on time according to the project plan. Another possible status request may be to report the number of change requests submitted, and of these submissions, how many were rejected and why.

5.2.2 Metrics

Metrics are a communication tool used by the project manager to assess control of the project and determine if process improvements are needed.

Metrics track what was changed, when it was changed, and what impact, if any, the change had on the previously captured data and potential effects on new data. These records should include enough detail to allow analysis and investigation of processes that may become unstable after a change was made. The optimum set of metrics and their scale of granularity depend on the organization strategies, resources, technologies, and business sectors. Some metrics may be measured in time scale or tolerances and others in cost. Each measurement can help the project meet the needs of its stakeholders.

Metrics help to estimate the effort required for CIs by establishing benchmarks for previous similar work efforts. These estimates of effort may help the project manager to identify trends in the productivity of implementing various CIs, determine if estimating techniques are valid, track customer satisfaction, and track rework. They can help to determine the impact a proposed change might have, and this may in turn aid decision making. This data may also help to facilitate the communication across the project, since decision-making data could be readily accessible by all.

For maximum utility, data may be presented to the project manager in a suitable format for enhanced analysis. This formatting may include numerical values, graphical representation, tables, or text.

5.2.3 Benchmarking

Project benchmarking involves comparing project processes, tools, and techniques with other projects. The other projects can be within or outside the project-owning organization. The Project Management Institute’s Organizational Project Management Maturity Model (OPM3®) is a source of best practices that can be used for comparison and measurement. Benchmarks are useful in planning PCM and in analyzing results.

Benchmarking is the process of capturing data from a functioning system. This data could be from a test system before it encounters any load, or it could be captured from a similar production system that a new application intends to surpass. PCM may also require a more specific type of benchmarking. Benchmarking can be prepared for any aspect of a system including: training time, learning curve, ease of use, as well as the normal data items of response time, customer serve time, and end-user satisfaction. For PCM, this type of benchmarking produces configuration items that are multipurpose. These items feed into the quality assurance processes and may trigger requests for changes such as corrective action based on variance analyses.

5.2.4 Performance Measures

Performance measures vary greatly depending on the type of application being developed. These measures can be expressed as Key Performance Indicators (KPIs) such as:

• Quantitative. This measure requires the definition of a set of objective metric(s) which can be expressed as a specific quantity such as an amount, sum, or number.
• Qualitative. This measure requires the definition of a project-specific set of non-computational attributes or characteristics.

5.2.5 Delivering Operational Statistics

Operational statistics can include Statistical Process Control (SPC) artifacts. Examples include control charts, run charts, feedback from users and customers, and overall determination as to whether the change met the desired result.

Operational statistics normally provide a measure of the repeatability and reproducibility required for determination of project stability over time. SPC can be used to measure process stability over time, and can provide a stable benchmark for auditing any data changes after change implementation.

5.2.6 Providing Reports

Status accounting provides either periodic or ad hoc reports. These reports are a communication tool that describes the change control activity and the status of the CIs. The reports may provide the following types of information:

.1 Baselines:

• when created and by whom;
• where the information is stored, and how to gain access to it.

.2 Change Requests:

• new since last report;
• active change requests and their status in the change process; and
• approved requests since the last report and action to implement;
• details of requests rejected

.3 Notifications:

• change notices and distribution for requested changes and the affected CIs; and
• changed CIs.

.4 Implementation Status:

• changes being implemented and their status; and
• audits planned.

5.3 Analysis

Analysis of process improvement metrics helps to determine if the project change management process is functioning as planned by improving the overall system. Trend data can be used to monitor performance levels. Measurement data variation may be related to any number of causes. Environment changes, the measuring procedure and equipment, and numerous other factors may impact the measurement data captured.