The ad hoc process level is unpredictable and often very chaotic. At this stage, the organization typically operates without formalized procedures, cost estimates, and project plans. Tools are neither well integrated with the process nor uniformly applied. Change control is lax, and there is little senior management exposure or understanding of the problems and issues. Because many problems are deferred or even forgotten, software installation and maintenance often present serious problems.

Although organizations at this level may have formal procedures for planning and tracking their work, there is no management mechanism to ensure that they are used. The best test is to observe how such an organization behaves in a crisis. If it abandons established procedures and essentially reverts to coding and testing, it is likely to be at the ad hoc process level. After all, if the techniques and methods are appropriate, then they should be used in a crisis; if they are not appropriate in a crisis, they should not be used at all.

One key reason why organizations behave in this fashion is that they have not experienced the benefits of a mature process and, thus, do not understand the consequences of their chaotic behavior. Because many effective software actions (such as design and code inspections or test data analysis) do not appear to directly support shipping the product, they seem expendable.

Driving an automobile is an appropriate analogy. Few drivers with any experience will continue driving for very long when the engine warning light comes on, regardless of their rush. Similarly, most drivers starting on a new journey will, regardless of their hurry, pause to consult a map. They have learned the difference between speed and progress. Without a sound plan and a thoughtful analysis of the problems, management may be unaware of ineffective software development.

Organizations at the ad hoc process level can improve their performance by instituting basic project controls. The most important are project management, management oversight, quality assurance, and change control. The fundamental role of the project management system is to ensure effective control of commitments. This requires adequate preparation, clear responsibility, a public declaration, and a dedication to performance. For software, project management starts with an understanding of the job’s magnitude. In any but the simplest projects, a plan must then be developed to determine the best schedule and the anticipated resources required. In the absence of such an orderly plan, no commitment can be better than an educated guess.

A suitably disciplined software development organization must have senior management oversight. This includes review and approval of all major development plans prior to their official commitment. Also, a quarterly review should be conducted of facility-wide process compliance, installed quality performance, schedule tracking, cost trends, computing service, and quality and productivity goals by project. The lack of such reviews typically results in uneven and generally inadequate implementation of the process as well as frequent over-commitments and cost surprises.

A quality assurance group is charged with assuring management that software work is done the way it is supposed to be done. To be effective, the assurance organization must have an independent reporting line to senior management and sufficient resources to monitor performance of all key planning, implementation, and verification activities. This generally requires an organization of about 3 percent to 6 percent the size of the software organization.

Change control for software is fundamental to business and financial control as well as to technical stability. To develop quality software on a predictable schedule, requirements must be established and maintained with reasonable stability throughout the development cycle. While requirements changes are often needed, historical evidence demonstrates that many can be deferred and incorporated later. Design and code changes must be made to correct problems found in development and testing, but these must be carefully introduced. If changes are not controlled, then orderly design, implementation, and testing are impossible and no quality plan can be effective.

The repeatable process has one important strength that the ad hoc process does not: It provides control over the way the organization establishes its plans and commitments. This control provides such an improvement over the ad hoc process level that the people in the organization tend to believe they have mastered the software problem. They have achieved a degree of statistical control through learning to make and meet their estimates and plans. This strength stems from using work processes that, when followed, produce consistent results. Organizations at the repeatable process level thus face major risks when they are presented with new challenges. The following are some examples of the changes that represent the highest risk at this level:

The key actions required to advance from the repeatable process to the next stage, the consistent process, are to establish a process group, establish a development process architecture, and introduce a family of software engineering methods and technologies.

The procedure for establishing a software development process architecture, or development life cycle, that describes the technical and management activities required for proper execution of the development process must be attuned to the specific needs of the organization. It will vary depending on the size and importance of the project as well as the technical nature of the work itself. The architecture is a structural decomposition of the development cycle into tasks, each of which has a defined set of prerequisites, functional descriptions, verification procedures, and task completion specifications. The decomposition continues until each defined task is performed by an individual or single management unit.

If they are not already in place, introduce a family of software engineering methods and technologies. These include design and code inspections, formal design methods, library control systems, and comprehensive testing methods. Prototyping should also be considered, together with the adoption of modern implementation languages.

With the consistent process, the organization has achieved the foundation for major and continuing progress. For example, the software teams, when faced with a crisis, will likely continue to use the process that has been defined. The foundation has now been established for examining the process and deciding how to improve it. As powerful as the process is, it is still only qualitative; there is little data to indicate how much was accomplished or how effective the process is. There is considerable debate about the value of software process measurements and the best ones to use. This uncertainty generally stems from a lack of process definition and the consequent confusion about the specific items to be measured. With a consistent process, measurements can be focused on specific tasks. The process architecture is thus an essential prerequisite to effective measurement.

The following key steps are required to advance from the consistent process level to the measured process level:

In advancing from the ad hoc process through the repeatable and consistent processes to the measured process, software organizations should expect to make substantial quality improvements. The greatest potential problem with the measured process is the cost of gathering data. There are an enormous number of potentially valuable measures of the software process, but such data is expensive to gather and to maintain.

Approach data gathering with care, therefore, and precisely define each piece of data in advance. Productivity data is essentially meaningless unless explicitly defined. For example, the simple measure of lines of source code per expended development month can vary by 100 times or more, depending on the interpretation of the parameters. Lines of code need to be defined to get consistent counts. For example, if one line brings in a routine with one hundred lines of code, should that be counted as one line of code or one hundred lines of code?

When different groups gather data but do not use identical definitions, the results are not comparable, even if it makes sense to compare them. The tendency with such data is to use it to compare several groups and to criticize those with the lowest ranking. This is an unfortunate misapplication of process data. It is rare that two projects are comparable by any simple measures. The variations in task complexity caused by different product types can exceed five to one. Similarly, the cost per line of code of small modifications is often two to three times that for new programs. The degree of requirements change can make an enormous difference, as can the design status of the base program in the case of enhancements.

Process data must not be used to compare projects or individuals. Its purpose is to illuminate the product being developed and to provide an informed basis for improving the process. When such data is used by management to evaluate individuals or teams, the reliability of the data itself will deteriorate.

The two fundamental requirements for advancing from the measured process to the next level are:

In varying degrees, process optimization goes on at all levels of process maturity. With the step from the measured to the optimized process, however, there is a paradigm shift. Up to this point, software development managers have largely focused on their products and will typically gather and analyze only data that directly relates to product improvement. In the optimized process, the data is available to tune the process itself. With a little experience, management will soon see that process optimization can produce major quality and productivity benefits.

For example, many types of errors can be identified and fixed far more economically by design or code inspections than by testing. Unfortunately, there is only limited published data available on the costs of finding and fixing defects. However, from experience, I have developed a useful guideline: It takes about 1 to 4 working hours to find and fix a bug through inspections, and about 15 to 20 working hours to find and fix a bug in function or system testing. To the extent that organizations find that these numbers apply to their situations, they should consider placing less reliance on testing as their primary way to find and fix bugs.

However, some kinds of errors are either uneconomical to detect or almost impossible to find except by machine. Examples are errors involving spelling and syntax, interfaces, performance, human factors, and error recovery. It would be unwise to eliminate testing completely because it provides a useful check against human frailties.

The data that is available with the optimized process provides a new perspective on testing. For most projects, a little analysis shows that two distinct activities are involved: removing defects and assessing program quality. To reduce the cost of removing defects, testing techniques such as inspections, desk debugging, and code analyzers should be emphasized. The role of functional and system testing should then be changed to one of gathering quality data on the programs. This involves studying each bug to see if it is an isolated problem or if it indicates design problems that require more comprehensive analysis.

With the optimized process, the organization has the means to identify the weakest elements of the process and to fix them. At this point in process improvement, data is available to justify the application of technology to various critical tasks, and numerical evidence is available on the effectiveness with which the process has been applied to any given product. An organization should then no longer need reams of paper to describe what is happening because simple yield curves and statistical plots can provide clear and concise indicators. It would then be possible to ensure the process and hence have confidence in the quality of the resulting products.