We can now use the calculations in the previous sections to provide a rough answer to this central question in project planning.

The project that motivated this analysis was a very large software project (over 10,000 KSLOC) that wanted to get the suppliers on the job quickly, at the risk of spending an inadequate amount of time in system architecture and risk resolution before putting supplier plans and specifications into their Requests for Proposals (RFPs). The project needed to determine its “sweet spot” between the risk of having the suppliers deliver incompatible components, thus incurring schedule overruns due to rework, and the risk of spending too much time on system architecting and risk resolution, which might not leave enough time for the suppliers to develop their system components. This section shows how the COCOMO II RESL factor results were used to determine an adequate architecting “sweet spot” for this project and other sizes of projects.

The full set of effects for each of the RESL rating levels and corresponding architecting investment percentages are shown in Table 10-4 for projects of size 10, 100, and 10,000 KSLOC. Also shown are the corresponding percentages of total delays in delivery, obtained by adding the architecting investment time to the rework time, assuming constant team size during rework in order to translate added effort into added schedule. The bottom two rows of Table 10-4 show that the added investments in architecture definition and risk resolution are more than repaid by savings in rework time for a 10,000 KSLOC project up to an investment of 33% in architecting, after which the total delay percentage increases. In short, the minimum-delay architecting investment “sweet spot” for the 10,000 KSLOC project is 33%.

The top graph in Figure 10-10 shows the results of Table 10-4 graphically. It indicates that for a 10,000 KSLOC project, the sweet spot was actually a flat region around a 37% architecting investment (as a result of this analysis, the project added another 18 months to its architecting schedule). For a 100 KSLOC project, the sweet spot is a flat region around 20%. For a 10 KSLOC project, the sweet spot is around 5%. Thus, the results in the graph and Table 10-4 confirm that investments in architecting are less valuable for small projects but increasingly necessary as the project size increases.

Recent further analyses of the COCOMO II database on the effects of requirements volatility and system criticality are shown in the bottom graph of Figure 10-10. Here, the solid black lines represent the average-case cost of rework, architecting, and total cost for a 100-KSLOC project, as shown at the left. The dotted red lines show the effect on the cost of architecting and total cost if rapid change adds 50% to the cost of architecture and risk resolution. Quantitatively, this moves the sweet spot from roughly 20% to 10% of effective architecture investment (but actually 15%, due to the 50% cost penalty). Thus, high investments in architecture, feasibility analysis, and other documentation do not have a positive return on investment for very high volatility projects, due to the high costs of analysis and documentation rework for rapid-change adaptation.

Figure 10-10. Penalties for architecture evidence shortfalls and resulting architecting investment sweet spots

The dashed lines in the bottom graph of Figure 10-10 represent a conservative analysis of the external cost effects of system failure due to unidentified architecting shortfalls. These estimates of the costs of architecting shortfalls include not only added development project rework, but also losses to the organization’s operational effectiveness and productivity. These are conservatively assumed to add 50% to the project-rework cost of architecture shortfalls. In most cases, for high-assurance systems, the added cost would be considerably higher.

Quantitatively, this moves the sweet spot from roughly 20% to over 30% as the most cost-effective investment in architecting and development of feasibility evidence for a 100 KSLOC project. As mentioned earlier, all the “sweet spots” are actually relatively flat “sweet regions” extending 5–10% to the left and right of the “sweet spots.” However, moving to the edges of a sweet region increases the risk of significant losses if some project assumptions turn out to be optimistic.

To summarize the lessons from Figure 10-10, the greater the project’s size, criticality, and stability, the greater the need for validated architecture feasibility evidence. However, for very small low-criticality projects with high volatility, the architecting efforts make little difference and don’t need to be redone continuously, which would produce a negative return on investment. In such cases, Agile methods such as XP will be more effective. Overall, evidence-based specifications and plans will not guarantee a successful project, but in general will eliminate many of the software delivery overruns and shortfalls experienced on current software projects.

As one final caveat, the sweet spot investment numbers in Figure 10-10 are not one-size-fits-all architecting investment numbers for projects to put into their budgets, but instead early indicators of the right planning range for the architecting investments. Once the project reaches the detailed planning and budgeting stage, other cost drivers—such as personnel capability and experience, application complexity, legacy migration complexity, technology risk, process maturity, and tool support—can also influence the appropriate level of architecting investment. A more recent Constructive Systems Engineering Model (COSYSMO) has been developed and calibrated to support a project’s more detailed determination of its needed amount of architecting investment [Valerdi 2005]; [Valerdi and Boehm 2010].