CHAPTER 2
Project Things Metrics
The quantitative facets of projects, also referred to as project “things,” are the visible, tangible signs of the implementation—and eventual success—of the project. Until the 1990s, most project management metrics systems focused primarily on the “things” attributes of projects, particularly aspects involving schedule, cost, and resource utilization. This outlook still predominates in organizations that have not advanced beyond the first two levels of a staged maturity scale.
However, since the 1990s, many new, more comprehensive project management tools and techniques have evolved. In fact, project things now represent just one of three sets of project attributes that need to be quantitatively planned and monitored. The other attributes deal with people and the enterprise.
Generally speaking, project performance indices provide feedback on activities that lead to the project scope and, to the extent to which these activities are aligned with project objectives, client expectations and the organization’s strategic objectives. It is fair to say that things metrics characterize tools that assess the progress and success of projects almost as if the project processes run by themselves without the intervention of people. However, to be accurate, the success of things issues reflects the success of project team members. Yet, when the focus of metrics is exclusively on things, the facets of the project measured are different. In essence, things metrics measure the health of a project. (Again, people and enterprise issues affect project health, but people and enterprise attributes are quantified separately.)
Things metrics quantify and measure cost, schedule, scope, quality, and the project’s overall success in meeting client needs. If the organization already has one of several variations (Rad and Levin 2002) of a Project Management Office (PMO), the project functions of the PMO will directly benefit projects (see Figure 2-1). The enterprise functions of the PMO will directly benefit the enterprise, but they will indirectly benefit projects as well.
Figure 2-1
Functions of the Project Management Office
The metrics relating to the project functions of the PMO, and even those relating to the enterprise functions of the PMO, have a major things component. Thus, most metrics that measure the sophistication or effectiveness of the project and enterprise categories of PMO functions also will quantify the success of project things issues.
CLIENT REQUIREMENTS
Requirements can be described as the articulation of a client’s wishes, which often are expressed in terms of performance, such as speed, error rate, or processing rate. Requirements also can be defined in terms of deliverables such as software, hardware, machinery, or structures.
The project scope document represents the team’s response to a client’s wishes. This response is usually expressed in the details of deliverable attributes and through the means and modes of crafting the deliverable. Deliverable attributes can take the form of physical size, mechanical features, software modules, and test performance targets. (Instrument 2-A) Metrics can quantify the formality and sophistication of responding to a client’s needs, as expressed in plans for managing cost, scope, quality, risk, communications, etc.
The description of project objectives should include attributes and characteristics of the deliverable, such as physical size, capacity, length, height, or strength. The objectives also should specify an identified level of performance, a prescribed level of quantified reliability, a critical speed to be attained, a quantified level of system availability, the ability to handle a given number of transactions within a defined period of time, or the ability to provide a certain level of quantified client satisfaction.
Other deliverables and objectives could include physical tolerance, physical speed, software tolerance limit, software processing speed, and software processing accuracy. Additional deliverable attributes could include indicators of surface texture, quantified robustness features, software error frequency, quantified measures of user friendliness of the software, and quantified personnel skills.
Usually, the information describing project requirements includes, and sometimes overlaps, the business case of the project and the scope definition of the project’s deliverable. Although requirements are client wishes that dictate scope and quality of the deliverable, requirements and scope are not synonymous.
The project manager must draw a clear distinction between the project’s business case and the description of the project’s deliverables (see Figure 2-2). Because there should be a close and distinct correlation between the project plan and the requirements, the project manager must be vigilant in removing any real or perceived inconsistencies between client requirements and the project plan. Then, the project manager must carry this a step further by distinguishing among deliverables, constraints, and assumptions. (Instrument 2-B)
During the life of a project, requirements will change for a variety of reasons. Accordingly, it is necessary to track differences between planned and developed requirements by assessing the status of each component of the deliverable as it moves through the life-cycle activities. In addition, metrics for requirement volatility, such as the percentage of requirements that are changed, can be prepared by maintaining a detailed history of the requirements changes and the rationale for each one. Finally, one reality of project work is the inevitability of defects in the deliverable and the concomitant need for rework to correct them.
Figure 2-2
Continuum from Strategic Vision to Project Deliverable
Thus, project progress documents must track activities conducted to make changes in the deliverable in order to resolve any physical or performance defects. (Instruments 2-C and 2-D) Further, there is a potential risk, especially in new product development or in systems development projects, that the resulting product will not perform fully in line with its intended purpose. Thus, the relevance and applicability of the deliverable must be continually verified.
During the early stages of the project, the project manager should prepare a statement of detailed project requirements for client approval and signoff. The frequency and nature of the requests for change must be tracked to assess the completeness of the process for analyzing the requirements and defining the scope. If both the requirements analysis process and the scope definition process are comprehensive, there probably will be fewer change requests during the life of the project.
In turn, a comprehensive scope definition will reduce the number of unplanned quality audits and project reviews. Thus, the audit should be considered an effective quality assurance tool for the project manager, rather than a “gotcha game” (Levin 1998). Using these proactive processes, defects detected during acceptance tests should decrease, as should the time and cost required to fix them.
Documents that record the evolution of project requirements and project scope allow direct or indirect tracking of the number of requirements changes made throughout the project. This tracking enables verification of whether changes were prompted by a need to define client requirements that were not detailed initially, or whether they can be attributed to significant changes in client requirements during the life of the project. Proper metrics also allow assessment of the number of meetings held with stakeholders to identify requirements and the extent of stakeholder participation, as indicated by the number of stakeholders who signed off on the project requirements.
Monitoring project requirements activities involves quantifying the progression of requirements from concept to formulation, to design, to testing, and finally to acceptance. This process ensures that there is always an approved and current set of requirements at any given point in the project life cycle, even when the client’s needs or the project environment changes. Also, defining and redefining project requirements in a methodical fashion increases the likelihood that the resulting product will meet the required functionality.
Metrics associated with scope definition address the nature and size of the project and possibly the level of complexity of the change control plans. The number of changes to project scope, the desired deliverable quality, assumptions, and constraints can form the basis for a suite of metrics. Further, the completeness of the scope statement should be monitored during the initiation stage of the project, perhaps even through the execution stage. As an indirect measure of overall project success, it is possible to devise a suite of metrics for timeliness in resolving ambiguities of the project scope and for the number of times that scope issues had to be escalated to the project’s executive management.
Finally, a project is not considered complete until the client officially accepts it. Therefore, client signoff on project deliverables must be obtained, signifying that most, if not all, client requirements were met with the deliverable. More important, this document should explicitly highlight the extent to which the deliverable meets the client’s current or stated needs.
Planning is an iterative process throughout the life of each project, and plans are expected to change as new information is acquired. Accordingly, it is important to assess whether templates used for planning are sufficient or if changes are needed.
Metrics can be formulated to assess plan revisions necessitated by changes to subsidiary plans, such as the Procurement Management Plan, the Staffing Management Plan, or the Quality Management Plan. Further, for purposes of future projects, it often is helpful to track the extent to which stakeholders were involved in the planning process. An additional metric, tracking the specific contributions of stakeholders in developing these plans, can be extremely useful in characterizing the project environment.
DETAILED PROJECT PLANS
The project scope document is the formal project description developed in response to the client’s needs. (Instrument 2-E) The Work Breakdown Structure (WBS) is a useful tool for graphically depicting project elements and providing a framework for the project (see Figure 2-3). If it is prepared to the appropriate level of detail, and particularly if it is oriented toward deliverables, the WBS can accurately describe the scope of work. In turn, a clear scope statement can serve as the basis for detailed project planning, methodical project execution, accurate responsibility assignment, and informed progress tracking.
Figure 2-3
Work Breakdown Structure
Information for preparing the WBS comes from the project’s scope statement, historical files of past projects, and other files containing the original and final project objectives of previous similar projects. Given that project information is normally developed on a gradual basis, the WBS should be treated as an evolving document that is updated throughout the project life cycle.
The WBS provides a common framework for conducting specific tasks related to project elements. When logically constructed, it enhances monitoring, controlling, and reporting processes. (Instrument 2-F) For example, a WBS facilitates the process of integrating project plans for time, resources, and quality. An effective WBS encourages a systematic planning process, reduces the possibility of omitting key project elements, and simplifies the project by dividing it into manageable units.
When the WBS is used as the common basis for scheduling and estimating, it facilitates communication among the professionals implementing the project. In fact, methodical use of the WBS ultimately leads to more effective schedules and estimates. Although the WBS is not a metric per se, it can be used effectively to enhance estimates and schedules. (Instrument 2-G)
To develop a WBS, the project must first be divided into three to nine components, with seven considered typical. In turn, each of these elements is divided into smaller, identifiable modules. This process is repeated until the project deliverable has been divided into many manageable, identifiable units. A metric can be established to assess the effectiveness of the WBS in facilitating project activities. In addition, a set of metrics can measure the effectiveness of developing a WBS template for use on future projects.
Similarly, in-house resources should be cataloged in a methodical manner by creating a Resource Breakdown Structure (RBS). The RBS tabulates the resources available to, or needed in, projects of a certain type. Developing the RBS involves dividing the pool of resources into entities specific enough that the RBS can serve as a shopping catalog for the resources needed to craft each WBS element.
The RBS is developed by first dividing all the project resources into major categories such as labor, material, equipment, and tools. Then, each category is broken down into its logical components until the lowest-level element is a specific identifiable specialty (see Figure 2-4). The RBS is not a metric per se, but it provides the foundation for and means by which methodical cost analysis can be conducted using logical categorizations and summaries.
Developing a detailed project estimate begins at the lowest level of the WBS. Here, the cost of each element is calculated by multiplying the quantity of each required resource by its unit cost, which is obtained from the RBS (see Figure 2-5).
This type of cost estimate is called a “bottom-up estimate,” and it is derived from detailed information contained in the WBS and RBS at the time of the estimate. This calculation clearly indicates the category of the resource, its intensity, and its duration. For example, if a project needs three chemists for a period of four days to create a chemical compound, then the category is chemist, the intensity is three workers, the duration is four days, and the effort is 12 worker-days. At a unit cost of $300 per worker per day, the cost is $3,600.
Figure 2-4
Project Resource Breakdown Structure
Figure 2-5
Bottom-Up Estimating
While total project cost could be computed by adding all these costs together, doing so is somewhat unwieldy and is not reflective of higher-level information that is easily obtained. Therefore, the next step involves moving up a level to determine—by simple addition—the total quantity of resources necessary for all elements on the next WBS level. The process repeats, proceeding from bottom to top, until each element of the WBS is tagged with the total resources required, grouped by resource category.
Once the calculations are extended to Level Zero, the project’s total cost has been determined, as has the cost of all intermediate elements of the WBS. In addition to cost, the resource utilization values of all intermediate components defined in the WBS are available.
As in any estimating method, it is important to check the estimate against experiential data and the subjective knowledge of project professionals. The first key question to ask is, “Is the estimate of total project cost reasonable?” Since inaccurate estimates often result from inadvertent omissions of key elements in the WBS and the RBS, correction of the estimate primarily comprises filling logical gaps in the WBS. In addition, refining the WBS, by way of enhancing the estimates of the current elements, adds precision to the project estimate. (Instrument 2-H)
The WBS also serves as the foundation for the network diagram and the project schedule. Before developing the project schedule, the project manager and project team must define the logical sequence of WBS elements in a precedence table. This allows the network to be populated with anticipated dates for starting and completing project components.
Schedule metrics can be based on the most detailed activities of the project or its aggregate components. Using WBS terminology, schedules can be developed for the lowest level of the WBS or any one of its intermediate levels.
A detailed schedule network is not a metric in and of itself. However, it provides the foundation for metrics that assess the pace and productivity of the project. Scheduling-related instruments deal with formalized procedures in developing a logical sequence for project elements, identifying the critical path of project execution, and developing a prediction for the project delivery date.
Combining the predictions for resource expenditure with resource cost and sequencing of activities provides a wealth of project information. The combination of these indicators provides a resource demand forecast and a cash flow forecast. Then, based on these results, and in light of organizational constraints for cash flow and resource availability, it is possible to adjust project plans by reducing resource demand or, in rare cases, making organizational adjustments in favor of project progress.
CONCEPTUAL PLANS
As discussed, the most accurate, reliable project estimate can be developed when all WBS elements have been identified with a reasonable degree of reliability and when the RBS has been defined with an adequate degree of specificity. However, the amount of information required for a detailed estimate will not be available until a major portion of the project has been designed and implemented. Nevertheless, project stakeholders must have indications of project cost and duration in order to approve initial project implementation.
Project authorization decisions necessitate a preliminary estimate, even though this estimate is compiled prior to many essential actions, including clarification of project objectives, definition of project scope, full articulation of requirements, clear definition of functions, and formulation of system architecture. Thus, a tentative, abbreviated project plan must be developed during the project’s inception in order to formulate a rough estimate of cost and duration for purposes of preliminary decisions. Then, as information regarding the project deliverable is enhanced, a more reliable project estimate can be developed. Depending on the organization, the first estimate is sometimes called preliminary, conceptual, or order of magnitude.
In the absence of detailed project information, project managers use a variety of tools and techniques to formulate estimates. These tools and techniques are usually based on models with proven success in previous estimating efforts on this or other projects. These models use mathematical expressions, ranging from simple to complex, to estimate cost, duration, and resource demands of one single activity, an assembly, or an entire project as a function of one or more input variables.
Estimating models compute the values of a set of dependent outputs as a function of the values of a set of independent inputs, which might initially be sparse in amount and low in accuracy. The selection of the technique depends on organizational policies, the project manager’s experience, and the amount of information available to the project manager at the time of the estimate. The techniques often used for preliminary project estimates are ratio, analogous, range, modular, and parametric (Rad 2002) estimating techniques (see Figure 2-6).
The timing of the estimate affects how much information is available and the degree to which the information is reliable. As a result, the accuracy of available information will determine the level of sophistication of the estimate.
When little project information is available, the only feasible tool is the analogous estimate, or the ratio estimate, which focuses on the entire project. When more information becomes available, then a parametric tool can be used to estimate the WBS elements down to Level Two or Level Three. Finally, when enough information is available, a detailed bottom-up estimate can be prepared (see Figure 2-7).
Figure 2-6
Techniques for Rough Estimates
The ratio estimating technique is premised on the existence of a linear relationship between project cost and duration and one or more basic features of the proposed deliverable. The basic features can be related to either physical attributes or performance characteristics. Although ratio estimating is deceptively simple, given an appropriate base of historical data, it can be a powerful tool in developing quick estimates for prospective projects. For example, experience has shown that the cost of major turbines and generators in a power generation plant is nearly 30% of the total cost of the plant. Likewise, in a construction project, the total cost of the project is twice that of the materials and embedded equipment.
Further examples include: high-level design of a systems development project is nearly 30% of the total cost of the project; only 20% of the cost and effort of a systems project is spent in coding; 75% of the cost of a systems development project is labor; and engineering design of a facility is nearly 8% of the project’s total budget. In addition, extensive effort has been devoted to developing a relationship between the estimated lines of code, or the number of function points, and the total cost of a systems development project. However, this ratio is highly specific to the operating system and the system architecture.
The terms “parametric estimating” and “modular estimating” refer to two estimating models that are essentially quite similar in application, principle, and underlying structures. They possess different names because they come from different industries. For purposes of this book, “parametric estimating” is used to refer to both these techniques collectively (see Figure 2-8).
Models used for software and systems development projects use some or all of the following data in arriving at a tentative estimate for the project: system complexity, system size, personnel skill, resource availability, specificity of project objectives, clarity of requirements, operating system features, environmental characteristics, and the extent of new technologies involved in the project (see Figure 2-9). Models used for construction and industrial projects use the following data to predict project cost and duration: industry and project type, capacity and quantity, external and usable size, overall weight, project location, and the extent to which novel materials, tools, and techniques are required for the project (Rad 2002) (see Figure 2-10).
Among the most useful models for making conceptual estimates of the cost and duration of a project based on previous similar projects are the three-quarter rule, the square-root rule, and their modified counterparts (Rad 2002; Gates 1976) (see Figure 2-11). These techniques are premised on the existence of a mathematical power relationship between the ratios of cost, size, and duration.
Figure 2-8
Modular and Parametric Models: Input Indices
Figure 2-9
Estimate Parameters: Systems Development Projects
Figure 2-10
Estimate Parameters: Construction and Industrial Projects
Even when the historical files contain only one similar project, the equations presented in Figure 2-11 allow development of a tentative estimate of cost and duration, respectively. In addition, these techniques presume that data for several similar projects will follow a straight line on a log-log scale.
Therefore, in cases where some historical data are available for cost and capacity, a graph similar to that shown in Figure 2-12 can be constructed in order to plot the existing data on a log-log scale. In turn, once a pattern line is drawn through the existing data, the cost of future projects can be estimated by using the pattern line. Likewise, Figure 2-13 can be constructed from existing data for cost and duration, and its pattern line can be used to predict the duration of future projects if their total cost is known.
PROJECT QUALITY
Quality refers to physical attributes when thinking of quality control of deliverables. However, quality also can refer to those non-physical attributes that directly or indirectly result in client satisfaction. The continuum between these two somewhat different concepts has been variously called quality control, quality assurance, quality management, and total quality management (TQM). In other words, although quality control refers to the process of monitoring the physical quality of products, TQM refers to good business practices that result in client satisfaction because superior products and services were delivered. (Instrument 2-I)
Figure 2-11
Analogous Estimating
Figure 2-12
Modified Three-Quarter Rule: Estimating Cost from Capacity or Size
Figure 2-13
Modified Square-Root Rule
The acronym TQM is sometimes used in service environments where no physical product is involved. For purposes of this book, quality refers to the physical attributes of the project components. However, planning for project quality should be more than a plan for spot-checking, or even full-scale measuring, of the physical quality of the end product. A formalized quality plan should include clearly stated criteria for each work package. In turn, the client should use these criteria during the evaluation and acceptance of the product or service. (Instrument 2-J)
For purposes of progress monitoring, review gates can be established at identifiable milestones to verify that the project deliverable meets its intended specifications before continuing further in the process. Quality metrics can be established to determine progress in meeting the objectives of these review gates, even before getting to the specific review gate. (Instrument 2-K) Metrics that fit this category could be used to track trends in defects in deliverable components, grouped by severity or status. The availability of these data will help the project team identify, research, and resolve quality issues in a timely manner.
The emphasis here should be on ways to ensure that the quality of the project deliverable is in line with client expectations. This is usually achieved by establishing an infrastructure that is conducive to producing consistently high-quality results through formalized procedures, relevant guidelines, and specialized training.
The literature identifies volatility of project scope and quality as the leading cause of implementation errors, and eventually, as the leading cause of variance from the project plan (Rad 2002). Thus, the success, or lack thereof, of scope and quality issues tends to overshadow project performance in other areas. Naturally, the client’s satisfaction with scope and quality will be enhanced by the team’s diligence in developing a detailed scope definition during the early stages of the project.
To that end, client satisfaction could be directly linked to the team’s success in implementing a methodical procedure by which the scope is verified, modified, enhanced, and finalized. With respect to performance in cost or schedule, the client’s perception of success is normally based on the original values, the final values, and the relative magnitude of the variance between the two. Given that some variance in cost and schedule is justified, it is only the unjustified portion that becomes the basis for judgments as to whether or not the project was a success relative to that specific attribute and by how much.
To achieve an in-depth understanding of client involvement and satisfaction, each client could be asked to complete a survey at the end of the project. This survey would address project performance in terms of all its requirements, the overall success in meeting those requirements, and the client’s general satisfaction with the project deliverables.
To minimize the cost of quality in projects, the components of the cost of quality must first be identified. (Instrument 2-L) The two major categories of cost are those related to (1) prevention of defects and (2) remedy of failures if prevention efforts are insufficient. The cost of prevention includes costs related to training, process improvement, modeling, qualifying, certification, and inspections. The cost of defects includes the costs of error analysis, rework, modifications, recalls, and retesting. The annual enterprise cost of quality, in any of the preceding areas, gives a clear indication of whether new efforts in quality are necessary. (Instrument 2-M)
CONTRACTS
For many organizations, specific contracts may be awarded for selected project work packages. For others, outsourcing is the primary way in which business is conducted. Organizations that outsource an extensive amount of their development and expansion work tend to regard projects as a subset of the contracts with which these products are commissioned. As such, project performance is often viewed in terms of the legalities of the contract rather than the deliverables and business objectives that first motivated the project.
A more appropriate approach is to consider a contract as a way of commissioning project deliverables. Then, a contract becomes the legal instrument by which an organization acquires products and services from an outside source.
For project purposes, a contract is an administrative mechanism by which the project is conducted by personnel who reside outside the corporate boundaries of the performing organization. Under this mindset, the contract becomes subordinate to the project, and performance is viewed in project terms rather than contract terms.
The project team, working in conjunction with the organization’s procurement department, can establish metrics to track each contractor’s progress in meeting schedule, cost, and technical performance goals. Project contract documents comprise two major types: administrative and technical. The administrative part deals with the legal responsibilities of both parties and the processes and procedures for enforcing various contract clauses. The technical part deals with the technical content of the project. The technical content of the contract, and issues that directly affect the implementation pace of the deliverable, should be the focus of the project management metrics dealing with contracts.
There are two basic types of project contracts (see Figure 2-14). The first is the fixed-price or lump-sum contract, which requires detailed specifications. Usually, the contractor offering the lowest price is chosen. In this mode of contracting, the prospective contractor offers to deliver the specific project deliverable for a fixed price. The contractor guarantees the fixed price and thereby assumes all financial risk in implementing that project. That is, of course, if the initial set of client objectives and project specifications is spelled out with sufficient detail and if the project environment remains reasonably stable during the life of the project. Under ideal circumstances, this type of contract gives the contractor incentives to avoid waste, reduce costs, increase productivity, and improve profits.
The second type of contract is a cost-plus contract, under which the contractor is reimbursed for costs incurred in performing project activities.
Figure 2-14
Types of Contracts: Expanded
The contractor is also paid a fixed or variable fee. There are variations of this type of contract in which the fee is tied to performance-based incentives arising from early delivery or cost-cutting measures. Regardless, the client assumes a large portion of the project performance risk—and the financial risk—of the project in this type of contract.
Because of the obvious shortcomings of these two basic forms of contracts, a third category has emerged that incorporates some of the best features of the two while minimizing their shortcomings. Under this mode of contracting, the contactor quotes a price for each unit of each activity or deliverable. For example, under a unit pricing scheme, the client and the contractor might agree that the contractor will be paid $100 for each specialist hour, $2,000 for each personal computer, $35 for each foot of pulled network wire, etc. Aptly, this type of contract is called a unit-price contract.
In many ways, a unit-price contract is a compilation of hundreds of fixed-price contracts, thus allowing the client to change the overall scope with some ease. Sometimes this type of a contract is called a time and material contract.
Contractor performance is measured with two somewhat separate sets of indices. The first deals with project performance in terms of cost, schedule, and quality. These performance indices can compare the actual values of these attributes with their planned values. These indices also can evaluate the sophistication of a contractor’s plans for successfully completing project activities and the contractor’s expertise in interpreting the technical requirements of the project.
The second set of indices deals with the contractor’s behavior in terms of responsiveness to a client’s request for minor scope changes, more timeliness in communications, and greater efficiency in resolving conflicts and ambiguities (see Figure 2-15). The availability of such indices, with or without a model, enables clients to provide feedback to contractors as to whether or not their expectations were met. In many ways, the people issues of the external project, such as a subjective perception of project success, are very similar to those of an internal project. Thus, many people metrics of internal projects are applicable to outsourced projects.
MONITORING PROJECT PROGRESS
Most project indices relate to the monitoring phase of the project, although the foundations for these indices must be formed during the planning phase. Metrics can be used to assess tradeoff decisions among the areas of scope, cost, and time throughout the life of the project. (Instrument 2-N)
Figure 2-15
Contractor Evaluation Indices: Example
One common example of such a tradeoff occurs when the team must determine whether a project’s schedule should be compressed in response to a desired early delivery date. If schedule compression is needed, the impact of compression on resource requirements and budget is of interest. Using the cost estimating data, a metric can be established to record the initial project cost estimate and the updated estimates as the project moves through its paces. The values provided by this metric can help improve the quality of future estimates on similar or related projects. (Instrument 2-O)
While it is important to track and monitor project progress, the tracking process can easily become an end in itself. Thus, a metric should be devised to determine the effectiveness of the processes used to track and control schedule and cost. (Instrument 2-P)
It is often useful to track the number of key milestones completed and missed. This can be done by baselining the project schedule and tracking milestones against the specific WBS element and control account identification number. Differences in the scheduled delivery date and actual delivery date can thus be tracked. This information can be used to determine why delivery dates were missed. Thus, preventive action can be taken on future similar projects, and sometimes on the current project if it is a large, complex one. (Instrument 2-Q) At the organizational level, the number of people who use the schedule for tracking and controlling can be assessed.
As the project progresses through its life cycle, it is imperative to assess the project’s rate of progress regularly. The earned value technique provides this crucial piece of information. The technique’s premise is that the client will be fully informed of the pace of project progress with a combined accounting of how much time and money have been spent so far and how many deliverables have been delivered to date.
Although the earned value technique was initially developed for monitoring the status of outsourced projects toward achieving the goals of the project, it can serve as an equally powerful tool in determining the rate of progress on internal projects. Therefore, computation of earned value must be integrated into the progress monitoring system of every project, internal or external.
In fixed-price contracts, the contractor is focused on a very specific deliverable for a specific payoff. Therefore, in cases involving fixed-price contracts, the concept of earned value is generally used to calculate the amount of incremental payment that is due the contractor.
By contrast, on projects that are conducted through cost-plus contracts, the objective of the contractor is compliance with the directions and instructions of the client, even if they are incremental, incomplete, and possibly inconsistent over the life of the project. In cost-plus contracts, the contractor is paid for expenses regardless of the amount of progress toward the stated or intended project goal. Thus, the earned value will compute the real progress of the project and the real cost of that progress, but it might not impact the contractor’s payment.
Nevertheless, the project’s final cost and duration can be predicted based on project progress as of any point in the project. Availability of such predictions at the early stages of the project can facilitate informed project decision making regarding increased resource allocation or termination.
The first step in the earned value process is to formulate a list of values during the project’s planning stages. These values are usually expressed in terms of the monetary value of each element. Then, at reporting milestones, progress credited to each constituent element is determined, using measurement of the quantified progress. If progress is not directly quantifiable, any of the crediting methods shown in Figure 2-16 can be used.
The total earned value will be the sum of the products of the value amounts and credited progress. The value that is earned for each WBS element can be determined by summing the progress made in each of the tasks for a particular WBS element. Thus, at any point during the life of the project, the amount of progress can be determined, as indicated by the earned value of the project, by summarizing the earned value of lower-level components along the WBS structure.
Figure 2-16
Progress Reporting for Non-Measurable Activities
During the early stages of the project, and for small projects, this process involves only a few elements at the first or second WBS levels. For fully developed projects, the process involves a large set of all the lower-level components of the project, extending to Level Four, Level Five, and even lower on the WBS.
Figure 2-17 symbolically depicts the procedure for establishing an earned value system for a deliverable element. It shows that, independent of what was supposed to have been completed by the time of the review, the elements under review have completed between 14% and 90% of their assigned target delivery value. Thus, if the weight of all these elements is the same, the total value for this project would be 37%; if this were a fixed-price contract, the contractor would be due 37% of the contract value.
PROJECT RISK
Risk management remains one of the more elusive aspects of project management. Although several schemas are available for the process, the nature of implementation varies substantially from organization to organization in response to unique environmental features.
Essentially, the risk management process involves activities dealing with identifying the risks, quantifying the impact of those risks on the project, and mitigating the influence of those risks on the project’s progress. (Instruments 2-R and 2-S) Risk metrics can be divided into metrics that categorize the nature of risk events, metrics that categorize the probability of occurrence of those events, and metrics that predict the impact of the risk events on the project’s outcome.
Figure 2-17
Value Earned in Deliverable Modules
Each project will have certain risks associated with it. A risk tracking form can help determine the number of risks identified, the risks that actually occurred, the effectiveness of the risk responses employed, and the actual impact of the identified risks on the project. These kinds of data can be incorporated into a risk project repository database, which in turn can help identify the types of risks that may be expected on certain types of future projects at certain phases in their life cycle. (Instrument 2-T)
For each identified risk, a response plan should be established to detail the specific response that should be employed if and when the identified risk occurs. Accordingly, a suite of metrics can be established for evaluating the effectiveness of risk response plans. The indices of this suite would quantify the number of workarounds required, the type of mitigation actions that were implemented, and the team’s ability to mitigate the risks with appropriate contingency plans. (Instruments 2-U, 2-V, 2-W, and 2-X)
An interesting advantage of a well-constructed risk metrics system is that it enables project team members to identify important events and trends as preludes to separating problems from opportunities. The availability of such information allows team members to exploit emerging opportunities and construct plans to mitigate problematic events.
Within the context of a project, it might first appear that the client and the project team define success differently. The client is usually focused on the physical attributes of the project deliverables, whereas the project team is concerned with the means and modes by which those deliverables are designed and crafted. Further, sometimes the definition of success can vary during different phases of the project. A responsive, timely deliverable can be a primary indicator of both client and team success.
From the vantage point of the client, project success is measured by how closely targets of scope, quality, cost, and duration are met. A WBS-like structure can be used to highlight the client-focused elements of project success (see Figure 2-18). Although their relative importance will undoubtedly vary from one organization to the next, it is possible to develop a general weighting scheme for these elements (see Figure 2-19).
Client-centric project performance metric systems include those that describe the various attributes of scope, cost, and schedule. Metrics useful for this purpose include those that monitor progress in terms of the quality and extent of deliverables, including variances in cost and schedule.
Figure 2-18
Project Success Indicators: Client View
Figure 2-19
Illustrative Project Success Indicators: Client View
PROJECT SUCCESS FACTORS
Project success factors, as viewed by the extended project team, can be divided into two major categories: those that deal with things and those that deal with people (see Figure 2-20). The major feature of things issues is that they are somewhat easy to quantify; therefore, they lend themselves to tabulation, plotting, and evaluation by a variety of metrics.
In addition, the mission of the project manager is to develop processes and procedures for effecting acceptable levels of client satisfaction, vendor satisfaction, and team morale. A detailed knowledge of such people attributes provides the foundation for the team to measure its success in managing cost, schedule, quality, and scope.
The things metrics most useful to the project manager, and possibly to the project team, tend to include operational issues, such as sophistication of processes, effectiveness of processes, and, ultimately, the impact of those processes on the project deliverable. Such indices allow all team members to maintain more effective control over costs and schedules by reducing risks, recognizing opportunities, and improving quality.
Overall, the team then has a better likelihood of achieving significant project objectives to the satisfaction of the client. Summing team performance in these areas provides a barometer of how well tasks were conducted. Figure 2-21 shows a suggested weighting scheme for success indices of a project. (Instruments 2-Y and 2-Z)
Figure 2-20
Project Success Factors: Team View
Figure 2-21
Illustrative Project Success Factors: Team View
Things metrics are the most familiar metrics available for characterizing projects. By and large, these metrics describe the project deliverable and the efficiency with which it is crafted. The deliverable, as characterized by the WBS, can be quantified and tracked with reasonable accuracy. The metrics for means and modes of delivery include those dealing with managing elemental cost, project cost, and project schedule. Finally, metrics dealing with project delivery issues include the sophistication of the project team in managing project risk, quality, and contracts.
INSTRUMENT 2–A
Project Charter
Strategic Objectives of the Project |
||
Business Case |
||
Project Scope and Deliverables |
||
Requirements |
||
Assumptions |
||
Constraints |
||
Assigned Project Manager and Level of Authority |
||
Participation by Functional Departments |
||
Major Risks |
||
Major Milestones |
||
Conceptual Budget |
||
|
• Project Sponsor • Project Manager • Project Team • Key Stakeholders |
||
Approvals: | ||
Adapted from Parviz F. Rad and Ginger Levin, Achieving Project Management Success Using Virtual Teams (Boca Raton, FL: J. Ross Publishing). © 2003 by J. Ross Publishing, Inc. Reprinted with permission.
INSTRUMENT 2-B
Requirements Documentation Form
Client Information |
Business Objectives of Project Deliverables |
Requirements |
Project Description |
Deliverables |
Project Boundaries |
Acceptance Criteria |
Acceptance Tests |
Constraints |
Assumptions |
Major Risks |
Major Milestones |
Approval Requirements |
INSTRUMENT 2-C
Defect Identification Log
INSTRUMENT 2–D
Defect Tracking Log
Use this instrument to record project team progress in identifying defects and in successfully resolving identified defects.
INSTRUMENT 2-E
Abbreviated Project Planning Checklist
Use this instrument as a quick, high-level reminder to perform all the tasks required during the planning phase of the project.
Documented business goals and objectives
Project scope statement
Major deliverables
Work Breakdown Structure (WBS)
Top-down cost estimates
Major milestones and target dates
Budget
Description of project management approach or strategy
Responsibility assignments for each WBS element
Performance measurement baseline for technical scope, schedule, and cost
Key or required staff and their expected cost and/or effort
Subsidiary management plans
• Change management
• Communications management
• Contract management
• Cost management
• Documentation
• Performance, evaluation, and test
• Process improvement plan
• Procurement management
• Quality management
• Resources
• Risk management
• Schedule management
• Scope management
• Staffing management
• Training
INSTRUMENT 2-F
Work Breakdown Structure Dictionary
Client: |
|
Project Manager: |
|
Team Member in Charge: |
|
WBS Number: |
Code or Account Idenfier: |
WBS Element: |
Revision Number/Date: |
Statement of Work: |
|
Responsible Organization: |
|
Names of Tasks Associated with This WBS Element: |
|
Contract Information (if applicable): |
|
Quality Requirements: |
|
Duration: |
|
Deliverables: |
|
Estimated Hours: |
Estimated Cost: |
Required Materials: |
|
Estimated Amount: |
Estimated Cost: |
Required Equipment: |
|
Estimated Amount: |
Estimated Cost: |
Technical References: |
|
Approvals |
|
Project Manager |
Date |
Responsible Team Member |
Date |
Stakeholder 1 (if applicable) |
Date |
Stakeholder 2 (if applicable) |
Date |
Stakeholder N (if applicable) |
Date |
INSTRUMENT 2-G
Work Breakdown Structure Development Checklist
Use this checklist during WBS development to ensure that items are not inadvertently omitted.
Divide the work into three to nine categories.
Divide each category into three to nine packages.
Divide each package into three to nine modules.
Continue to divide as long as logically possible.
Ensure that the numbering schemas for WBS elements represent the hierarchical structure.
Recognize that not all branches need go to the same level, but the significance of all the lowest-level items in the overall project should be similar.
Ensure that deliverable elements are logical, unique, and distinct.
Ensure that each WBS element at the lowest level represents a single, tangible deliverable.
Ensure that each intermediate WBS element represents an aggregation of all subordinate WBS elements listed below it.
Recognize that the level of significance of all lowest-level elements should be the same.
Refine the WBS on a regular basis as the project team develops more information about the deliverables.
Develop the WBS Dictionary.
Update the WBS when any work scope change occurs.
Update the scope statement, if required, based on development of the WBS.
INSTRUMENT 2-H
Effectiveness of the Work Breakdown Structure
The overall level of utility of using a Work Breakdown Structure (WBS) and the WBS Dictionary is the average of the scores reported for the items below. Using feedback on performance on current or past projects can help improve the chances of success on future projects.
1—Not at all
2—Marginally
3—Partially
4—Mostly
5—Fully
INSTRUMENT 2-I
Importance of Quality Initiatives to the Enterprise
Use this form to evaluate the degree to which the organization places emphasis on quality initiatives. Regular evaluation should provide occasions for celebration, as ratings should increase with the benefit of feedback.
Rate the following in terms of its importance in the organization:
1— Not at all
2— Marginally
3— Partially
4— Mostly
5— Fully
INSTRUMENT 2-J
Quality Assurance Survey
Assess the following factors and determine their level of importance with regard to the project:
1— Never
2— Rarely
3— Sometimes
4— Most of the time
5— Always
NA—Not Applicable
INSTRUMENT 2-K
Quality Management Planning Checklist
Use this instrument to guide the project’s quality management aspects. While users need not follow the instrument sequentially, unaddressed items might indicate an opportunity for improvement in future projects.
Overall Quality Management Aspects
Project Scope
• Describe the project; include the scope statement or provide a summary description of the overall project, its objectives, the customer, a summary of the requirements, and the acceptance criteria.
Project Quality Policy
• Define the project’s quality policy.
Quality Assurance Manager
• Identify the Quality Assurance Manager for the project and state whether this person is assigned to the project full or part time.
• Provide a summary of this Quality Assurance Manager’s responsibilities.
• Discuss the relationship of the Quality Assurance Manager to the Project Manager.
• Discuss the relationship of this manager to the Quality Control Manager.
Quality Control Manager
• Identify the Quality Control Manager for the project and state whether this person is assigned to the project full or part time.
• Provide a summary of the Quality Control Manager’s responsibilities.
• Discuss the relationship of the Quality Control Manager to the Project Manager.
• Discuss the relationship of the Quality Control Manager to the Quality Assurance Manager.
Project Quality Team Responsibilities
• Identify the responsibilities of the project quality team as a whole.
• List any specific task-related quality responsibilities, such as responsibility for acceptance tests or audits and use of checklists, benchmarking, cost-benefit analysis, and experiments.
Cost of Quality
• Specify the estimated quality costs in terms of prevention costs, appraisal costs, and failure costs.
Quality Management Plan Review and Update
• Describe how quality assurance and quality control will be handled on the project.
• Specify how often the Quality Management Plan will be reviewed and when it will be updated during the project.
• Specify who will receive copies of the Quality Management Plan and how it will be disseminated.
Quality Metrics
• Define the specific quality metrics that will be used in quality assurance and quality control.
Quality Baseline
• Establish the project’s quality baseline.
• Describe how updates to the quality baseline will be handled.
Process Improvement Plan
• Describe the process improvement plan to be followed to ensure that non-value-added activities are not performed.
• Discuss how quality improvement will be identified.
• Describe how lessons learned will be collected and documented.
Quality Assurance of Project Deliverables
Deliverable Description
• Describe the project deliverables.
Test and Acceptance Process
• Discuss any planned test and acceptance processes and how they will verify the quality of the deliverable items.
Acceptance Criteria
• Describe the acceptance criteria for project deliverables and how these criteria will be used to assess project quality.
Project Audits and Quality Reviews
• Discuss the planned schedule of project audits and quality reviews.
• Discuss how audit results and review outcomes will feed back into project planning activities.
• Discuss how lessons learned will be identified.
Tools and Techniques
• Discuss how process analysis will be used to examine any problems, constraints, and non-value-added activities.
• Discuss any other quality assurance tools and techniques that will be used.
Change Management
• Describe how changes will be handled based on identified quality improvements.
• Describe how quality standards will be updated based on identified quality improvements.
Project Processes and Quality Control
Project Monitoring and Quality Control
• Discuss the in-process monitoring and control processes planned for the project and how they will affect project quality.
• Discuss how project-related data concerning quality will be gathered and used to control project quality.
• Discuss how quality improvements will be identified.
In-Process Quality Monitoring and Inspections
• Discuss how project quality will be measured as part of each work process in the project.
• Discuss the use of inspections as a quality control tool and technique.
• Discuss how deliverables will be validated.
• Discuss other quality tools to be used.
Change Management
• Describe how changes will be handled based on the results of quality control measurements.
• Describe how the quality baseline will be updated.
• Describe how other aspects of the project will be updated, such as standard checklists.
• Describe how lessons learned will be documented.
INSTRUMENT 2-L
Quality Cost Analysis Form
Use this instrument to document the cost of maintaining quality deliverables for the project.
INSTRUMENT 2-M
Project Quality Checklist
Use this instrument to record a summary of the activities involved in managing the quality issues of a project. Since this instrument yields primarily numeric results, the comparison of two projects will show which one is more sophisticated and will likely be indicative of future performance.
Number and type of revisions to the Quality Management Plan
Number of stakeholders involved in preparation of the Quality Management Plan
Number of problems discovered through quality inspections
Amount of rework required
Number of inspections conducted
Number of lessons learned and areas of potential improvements discovered through quality audits
Completeness of quality management activities compared to the plan
Efforts and funds in quality activities as compared to the plan
Extent of processes established to verify that quality improvements identified on each project are in fact implemented in the enterprise
INSTRUMENT 2-N
Change Order Request Form
Date: |
||
Project Name: |
||
Project Manager: |
||
Person Requesting the Change: |
||
E-Mail: |
Phone: |
|
Identify the Category of the Reason for the Change: Business Strategy Customer Request Design Philosophy Project Environment Technology Used for the Project Project Implementation Efficiency Other (please identify) |
||
Describe the Change: |
||
Scope |
||
Quality |
||
Project Duration |
||
Project Cost |
||
State the Actions Necessary to Implement the Change: |
||
State the Date by Which Implementation of the Change Is Required: |
||
Revise Budgeted Hours: |
from |
to |
Revise Budgeted Equipment and Materials: |
from |
to |
Revise Completion Date: |
from |
to |
Revise Scope and Quality: |
from |
to |
INSTRUMENT 2-O
Project Change Management Checklist
Use this checklist as a guide for managing changes in project attributes and circumstances.
Overall Project Status
What are the current values for project schedule, cost, and performance?
What are the current values for schedule, cost, and performance objectives of the individual work packages?
Is the client pleased with the results of the project?
If there are significant variances in the project performance, can these variances be remedied with
• Additional personnel?
• Reallocated personnel?
• Additional funds?
• Changes in deliverable attributes?
• Changes in schedule?
If there are positive variances, celebrate them and acknowledge the team’s efforts.
Project Duration
Is an overrun of the current duration baseline projected?
What is the cause of the variance?
Is a delay acceptable to the client?
If a delay is not acceptable, can additional resources be acquired?
What are the cost implications of additional resources?
Is additional funding the only way to maintain the original delivery date?
If a delay is acceptable to the client, will the delay result in an improved deliverable?
Project Cost
Is an overrun of the current cost baseline projected?
What is the cause of the variance?
Is a cost increase acceptable to the client?
If a cost increase is not acceptable, can the project scope and/or schedule be modified?
If a cost increase is acceptable to the client, will the increased cost result in an improved deliverable?
Is a shortfall of the current scope and quality baseline projected?
What is the cause of the variance?
Is a deliverable shortfall acceptable to the client?
If a variance is not acceptable, can additional funds be acquired, or can the project be delayed?
If a scope shortfall is acceptable to the client, will the shortfall result in improved cost and schedule performance?
INSTRUMENT 2-P
Project Status Report
Use this instrument periodically to record the status of project activities supporting accomplishment of stated goals.
INSTRUMENT 2-Q
Project Management Checklist
Use this instrument as a guide for performing all the necessary tasks for each phase of the project. The extent to which items are deliberately skipped might signal an opportunity for future enhancement.
INSTRUMENT 2-R
Risk Identification Checklist
Use this instrument as the basis for categorizing project risks. Use these categories as a first approximation, then enhance and modify them to be in line with the needs and policies of the organization.
Categorize the risks into categories such as:
• Business Impact—risks associated with constraints imposed by management or the marketplace
• Client Needs—risks associated with changes in attributes of the deliverable as suggested by the client or in client-imposed dates that affect the deliverable
• Client Characteristics—risks associated with the sophistication of the client and the Project Manager’s ability to communicate with the client effectively
• Deliverable Size or Capacity—risks associated with the overall size or capacity of the product to be built or modified and the expected quality level
• Development Environment—risks associated with the availability and quality of the tools to be used to build the product
• Legal and Regulatory—risks associated with possible legal or regulatory changes that may affect the project
• Organization—risks associated with the performing organization, such as those involving dependencies with other projects, resources that are supporting multiple projects and are needed at specific time frames, availability of funding, and project prioritization within the organization
• Project Management—risks associated with project management activities such as estimating, planning, scheduling, controlling, and communicating, with an emphasis on constraints and assumptions
• Staff Size and Experience—risks associated with the overall technical and project experience of the people who will do the work, particularly as external forces affect the size and nature of the project team
• Technology—risks associated with the complexity of the system to be built or product to be delivered, the inevitable rapid changes in the technology, and the extent to which the latest technology must be incorporated into the deliverable
INSTRUMENT 2-S
Characterizing Project Risk
Use this instrument as a first-order guideline for categorizing project risks. Note: There are several ways to categorize risk. Pick a strategy that is compatible with the organization.
1. Source of Risk
a. Within the project
b. Outside the project
c. Outside the enterprise
2. Probability of Occurrence
a. Qualitatively
b. Quantitatively
3. Impact of Occurrence
a. On schedule
b. On cost
c. On scope
d. On quality
e. On the project team
4. Response Plan
a. Avoid
b. Accept
c. Transfer
d. Mitigate
e. Exploit
f. Share
g. Enhance
h. Contingent responses
INSTRUMENT 2–T
Risk Tracking Form
Use this form to record and monitor the distribution of risk among the Work Breakdown Structure (WBS) elements. Cumulative entries will show the progression of this ongoing activity over the life of the project.
INSTRUMENT 2–U
Risk Ranking Form
Use this instrument to identify and prioritize project risks. Review for correctness on a regular basis.
INSTRUMENT 2-V
Risk Monitoring and Control Checklist
Use this instrument to record the intensity and sophistication of the risk activities of the project. Since there is no weighted scoring, and this is a first approximation, a general summary of positive/affirmative responses is sufficient as the rating. An absolute ranking is not as significant as an increase or decrease in ranking during two successive assessments.
INSTRUMENT 2–W
Risk Response Plan Tracking
Use this instrument for each risk in the risk response plan to track the effectiveness of the plan.
INSTRUMENT 2-X
Risk Map
Use this instrument to develop a tabular and visual map of project risks. Often, visual display of data enhances the decision-making process.
Classify each risk:
1—Negligible |
If the risk does occur, the project management team will passively accept it and not take any action. |
2—Marginal |
If the risk does occur, the project management team will actively accept the risk and use a contingency plan that it has developed. |
3—Probable |
The risk is likely to occur, but the project management team plans to avoid it to protect the project objectives from its impact. |
4—Critical |
The risk is expected to occur and, when it occurs, it is expected to affect the project objectives. Financial exposure is expected, and the project team has a transfer response prepared, which it will execute. |
5—Catastrophic |
The risk is expected to occur and, when it occurs, it will adversely affect the overall project objectives. A risk owner has been identified and risk triggers have been established. A mitigation response has been prepared. Contingency funds are available. |
Event Occurrence |
Event Impact/Probability |
Unlikely |
Not expected to occur unless there are unusual situations |
Possible |
Expected to occur once or twice during the project |
Probable |
Expected to occur several times during the project |
Likely |
Expected to occur often during the project |
Highly Likely |
Expected to occur regularly throughout the project |
Then, over time, prepare a risk index for each project. The Project Management Office can use the index at the enterprise level to assess the types of risks that have occurred and are expected to occur and to reevaluate the classification scheme.
Count the number of occurrences in each risk class.
INSTRUMENT 2-Y
Brief Post-Delivery Project Audit
Use this instrument to assess the sophistication of the project management activities, as a prelude to enhancing project procedures for future projects. The score for each area is simply the sum of positive/affirmative responses. With continuous improvements in project procedures, the audit scores should continue to increase.
INSTRUMENT 2-Z
Abbreviated Project Closeout Checklist
Use this instrument as a quick guide for the actions that need to be performed as part of the project closeout.
