CHAPTER 9
Applying Complexity Thinking to Large, Long-Duration Projects

	Independent	Moderately Complex	Highly Complex
Time/Cost	< 3 months < $250K	3-6 months $250K-750K	> 6 months > $750K
Team Size	3-4 team members	5-10 team members	> 10 team members

In this chapter we first explore the nature of project complexity on large, long-duration projects, which often span multiple years. We then recommend techniques for you to consider in leading your team to manage the project and deliver the most innovative solution.

WHAT MAKES LARGE, LONG-DURATION PROJECTS COMPLEX?

Of the various elements that combine to make long-duration projects complex, the most significant is the inevitable changes that will occur in the business environment, which will necessitate adjustments to virtually all elements of the project approach. Project size matters as well, with research demonstrating that the smaller the team, the greater the likelihood of success. In addition, team fatigue and burnout lead to complex human interactions and unavoidable staff turnover.

CONSTANT CHANGE

History is littered with large, long-duration projects that have failed to deliver (e.g., the Denver Airport, Boston’s Big Dig). One of the biggest problems with long-term projects is that so many unforeseeable things can happen. No matter how hard we try to stabilize requirements, complaining that the customer keeps changing its mind, the fact is, change happens. If project teams cannot adapt, these long-duration projects run the risk of working to achieve a business objective that is no longer valid or relevant: The new business solution the team is building simply may not meet current business needs. Dependencies on other projects or external constraints that have been identified and managed may disappear, but new ones will emerge. Because of constant change and uncertainty, long-duration projects in general elicit a lack of confidence in time and cost estimates and often an inability to deliver anything of value.

SIZE MATTERS

The Standish Group’s research has identified three key metrics that can predict with a high degree of certainty whether a project will be successful: project size, project duration, and team size. CHAOS research confirms that small projects are more likely to succeed than large projects.¹ However, most business transformation projects are large and long, involving an enormous amount of work. We recommend using an iterative, incremental approach to managing a long-duration project, to limit the dependencies and to structure the solution into small, manageable increments.

TEAM FATIGUE AND STAFF TURNOVER

Contemplate for a moment the trials and tribulations of a four-person crew crowded into a small capsule on a space flight. Long-duration interplanetary flight poses enormous challenges physically, socially, and psychologically. In prolonged space flight, a distinctive group culture will develop and the crew members will need strong coping mechanisms and a high tolerance of enforced social associations. Personality characteristics resilient against social burnout and psychological stress may be as important to a successful mission as the design of the mission vehicle itself. ²

The situation flight crews find themselves in is not unlike that of project team members placed on a new, long-duration project where the workload is extreme, the project mission is critical, and there is no end in sight. The stress levels caused by forced interactions can elicit a reaction from the nervous system similar to a mild version of the alarm response to danger. When team members are commingled for a prolonged period, they may experience exhaustion that can trigger depression, insomnia, attention deficits, mood instabilities, and even sometimes immune system pressure that results in physical illness. Long-term projects tend to expose our weakest links as our ability to endure extended periods of heavy workload conditions are stretched to the limit.³

The human element adds a great deal of complexity to long-duration projects. Human behavior is unpredictable. Team members are dependent on each other for success. Intricate associations develop among team members, adding invisible and unknown dependencies. Likewise, interrelationships develop between team members and stakeholders external to the project team. The informal communication networks and alliances that emerge are unpredictable and unmanageable.

DEALING WITH THE COMPLEXITIES OF LARGE, LONG-DURATION PROJECTS

The complexities of long-duration projects require that special attention be directed to planning and structuring the project, developing and delivering the solution, selecting team members, and sustaining a high-performing team over the long haul.

PLANNING AND STRUCTURING THE PROJECT

Our recommendations for planning and structuring large, long-duration projects include:

• Select the appropriate management approach

• Progressively elaborate the plan

• Use a systematic, reliable approach to estimating

• Perform rigorous time and cost management

• Use stage-gate management

• Conduct rigorous risk management.

Select the Appropriate Management Approach

Rigorous enterprise analysis should be performed during the pre-project study phase or immediately following project initiation to clarify high-level management issues and help the sponsor, customer, business analyst, architect, and project manager make the appropriate management choices for the project. The project leadership team should determine the specific nature of the business problem, appropriate management approach (e.g., linear, iterative, adaptive), and team structure. The team should work to answer questions like: Is this really a program? Is it a series of modestly scoped, small projects? Something else? Must the project or program deliver a product line, a system of systems? Can the solution be delivered in components?

Especially for long-duration projects, success depends on selecting the management approach best suited to deal with the changes that will inevitably occur. The project leadership team needs to (1) recognize the nature of the problem and solution, (2) understand that the conventional, reductionist systems/software engineering and project management approaches may not work, and then (3) make the right choice of management approaches (e.g., conventional versus adaptive techniques, appropriate project cycles, the best project team structure). Continuous customer and end-user evaluation and feedback should be built into the approach to ensure that the project team delivers what is needed—which often is not what was originally proposed.⁵

Progressively Elaborate the Plan

Progressive elaboration is defined by the PMBOK® Guide as a technique for continuously improving and adding detail to the plan as more information becomes available. The goal is for more accurate and complete plans to emerge from the successive iterations of the planning process. Also referred to as rolling wave planning and just-in-time planning, the detailed plan covers just a few weeks or months, and only high-level milestones are identified for the rest of the project. The plan thus evolves and adapts to changes as they occur.

Instead of trying to plan the entire project, start by scheduling the activities to define firm basic requirements.⁶ At the same time, begin to plan activities to develop a conceptual design of the solution at a high level, resisting design decisions that will impose constraints. Define the remaining components only at a milestone level (the PMBOK® Guide defines a milestone as a significant point or event in the project), based on the project cycle you have selected (as described in Part III). Sometimes it is advisable to set up a proof-of-concept pilot project to build small-scale prototype components of the solution and plan for interim decision points before launching into the full-blown project. Other times, you should separate a high-risk component of the full solution (e.g., the baggage system for Denver International Airport, which caused huge delays and cost overruns) and treat it very differently from the rest of the project.⁷

After basic requirements are understood and considered firm, and the team has an initial concept of what the solution will be, you will be able to plan the design, construction, and test activities in more detail. This approach makes it possible to request funding and the resources needed in increments instead of all at once.

Use a Systematic, Reliable Approach to Estimating

According to the Standish Group, although 28 percent of IT projects are coming in on time and on budget, most of those projects were originally overestimated. In a number of focus groups, “IT executives reported that they first get their best estimates, multiply by two and then add a half. It should not be surprising, therefore, that the majority of these successful projects were already 150% over budget before they began.”⁹ The report goes on to state the obvious: Reliable estimates are “rare birds” and our current estimating methods for IT projects are outmoded and ineffective.

Nonetheless, we must strive to come up with the best estimate we can. In fact, one precondition to being assigned as manager of a complex project should be a track record of developing reliable estimates. To increase reliability, we recommend using multiple estimating techniques. In addition, educate your project sponsor and other key stakeholders about the fallibility of estimates in general and discuss the reliability they can expect from your specific estimates.

Using progressive elaboration planning techniques, your near-term estimates derived from a detailed schedule should be quite reliable, within +/-10 percent (bottom-up estimates). But for out-phases where detailed planning has not yet occurred and the complete picture has not yet come into view (top-down estimates), it is wise to give a range for your cost and time estimates (e.g., $500K-$750K, four to six months) and a reliability designator (e.g., 75 percent probability).

Without a doubt, early estimates will be highly unreliable, exhibiting a wide range of variability. Numerous uncertainties are involved when building something unique with a team that has not worked together in the past. However, once the project has executed through a few iterations (if using incremental techniques) or through a few project phases (if using linear techniques), you can begin to gauge the speed of progress and adjust your original estimates accordingly. In Estimation for Agile Projects, Mike Griffiths presents a thorough discussion of estimating best practices that can be applied to virtually all projects.¹⁰

A variety of cost estimating techniques can be used. In an uncertain world, involve numerous stakeholders, use multiple techniques to confirm the reliability of your estimates, and document the assumptions and methods you used to come up with them. Use expert judgment and historical information from similar projects to help devise and verify early estimates. It may also be helpful to use industry guidelines to create estimates.

There are two broad categories of estimates, heuristic and parametric:

• Heuristic: based on expert judgment

  • Top-down estimates, also referred to as activity-based estimates—estimates are calculated using a broad, high-level deliverables list; they involve generating estimates for large units of work.

  • Bottom-up estimates, also referred to as task-based estimates—estimates are calculated using a complete, detailed work breakdown structure (WBS); they involve generating estimates for small units of work. Build a WBS and estimate the time and cost associated with lowest-level activities for near-term project phases.

  • Work distribution estimates—effort is estimated by major project phase.

  • Comparison estimates—effort is compared with completed similar project(s) and adjusted for dissimilarities.

• Parametric: based on calculations

  • Function point estimates—estimates are developed for logical business views of components of the solution (e.g., for each function: inputs, outputs, tables, queries, and interfaces).

  • Use case point estimates—estimates are developed based on the number of use cases.

Developing good estimates is difficult for a variety of reasons. There is usually intense political pressure to make the estimate what the project sponsor or the organization wants to hear to get the project approved. There is also a psychology to estimating—we are hopelessly optimistic and always think we can do more than is possible. And then there are technical issues; for example, projects involving software development simply defy estimation.

Lev Virine and Michael Trumper offer a few golden rules of estimating that they call “simple remedies”:¹²

• Never make a wild guess

• Collect relevant historical data

• Perform reality checks

• Conduct an independent assessment.

Perform rigorous Time and cost management

Delivering on schedule is one of the main challenges for a long-duration project, simply because of the enormous amount of work to be accomplished. Implement a rigorous process for tracking progress and controlling output. Track progress to the next milestone scrupulously. Manage the schedule and budget by establishing a project support team to update and maintain the schedule and budget baselines; emphasize to team members that they should bring any issues that put the next milestone in jeopardy to your attention.

Use Stage-Gate Management

Stage-gate management can be used to create opportunities to gather feedback from your customers and your team members on a regular basis. After completing each phase or iteration, conduct informal team-based quality reviews of deliverables. As part of these reviews, determine what worked well and identify opportunities for improvement to the solution development process and team operations. Subsequently, conduct a formal external quality assurance review of major deliverables and incorporate actions to correct defects found in the deliverables that must be resolved before work can proceed. Update the project cost, schedule, and scope baselines for the remaining near-term project phases/iterations, incorporating lessons learned into the plans.

Simultaneously, reexamine the business case to validate that business benefits will be achieved and the investment is still sound. Conduct a formal project review with the project sponsor and other key stakeholders to secure approval to formally launch and expend funds for the next phase/iteration.

Conduct Rigorous Risk Management

Few projects perform adequate risk management. For large, long-duration projects, it is essential to identify risks after each iteration/phase and reexamine risk responses to:

• Ensure the risk response plans are managing known risks Identify new risks and develop risk response plans

• Identify new project dependencies and interrelationships and develop dependency management plans.

• Chapter 16 focuses on risk management as it relates to the management of complex projects.

DEVELOPING AND DELIVERING THE SOLUTION

We offer a few suggestions for developing and delivering the solution on large, long-duration projects:

• Structure your project to develop and deliver the solution incrementally

• Minimize scope

• Delay design decisions until the last responsible moment

• Use rapid application development

• Use lean development techniques.

Structure Your Project to Develop and Deliver the Solution Incrementally

Research has repeatedly demonstrated that short-duration projects are more likely to be successful than prolonged endeavors.¹³ However, business transformation projects tend to be long and to involve a mix of complex development efforts, such as business process reengineering, legacy IT system replacement, and the creation of new, innovative business practices that rely heavily on technology.

To increase the probability of project success, structure your project into multiple deployments of small solution components rather than taking the “big bang” implementation approach. As you develop and deliver the solution in increments, incorporate lessons learned from each increment into the next iteration and constantly test for alignment with business objectives. This technique establishes a cycle that builds, refines, and reviews, enabling the correct solution to emerge gradually. The project is divided into phases or iterations with clear objectives that culminate in a project decision gate, where progress and risk are reviewed and plans are set for the next phase. The first phase is typically an effort to define basic requirements and experiment with solution design options aimed at resolving uncertainties about the feasibility of technical options. This phase may be followed by a series of prototyping iterations that gradually bring the solution into view. This technique can be difficult to control, but it is highly effective when properly applied.

The Standish Group Recipe for Project Success (Table 9-1) asserts that “success is practically in the oven” when a project follows this recipe. Standish reports that the newest data coming in suggest that it is prudent to further reduce the amount of resources (people, time, and cost) to no more than four people, for no longer than four months, at a cost of less than $500,000. For large, long-duration projects, the only way to get the resources down to this level is to structure the effort into a program comprising multiple projects and to use incremental/iterative solution delivery.

TABLE 9-1. Standish Group Recipe for Success, 2001¹⁴

Ingredients	Clear business objective; minimized scope (microprojects with rigorous configuration management); constant communication and collaboration; proven, standard, stable software infrastructure (versus custom code); firm basic requirements; formal methodology; reliable estimates
Mix with	Full-time, co-located core team members (experienced business analyst, project manager, business visionary, architects, and developers) coached by an involved executive project sponsor, involved stakeholders, an iterative development process, and effective decision-making tools (requirements tools, project management tools, design/analysis tools, and modeling tools)
Bake	No longer than six months; no more than six people; at no more than $750,000 (1999) No longer than four months; no more than four people; at no more than $500,000 (2001)

Minimize Scope

The motto of 21st century projects is: “Barely sufficient is enough to move on.” The more features and functions, the larger the project; as we have discovered, less is more.

So, how does minimizing scope really work? Initially, deliver a minimum viable subset of the full solution to start adding value for the organization as early as possible. Then, continue to deliver components of the system in short-interval deployments. Limit the dependencies between solution components to reduce the cost of changes. Design the solution to be flexible and agile to allow the customer to respond to changes in the business need, technology, or market conditions.

Delay Design Decisions until the Last responsible moment

Flexibility comes from delaying design decisions and the start of major activities for key project drivers (information flows, technical decisions, and business decisions) until the last responsible moment, that is, the latest moment possible without compromising cost or schedule. This “keep your options open” approach allows for maximum flexibility.¹⁶

Use Rapid Application Development

Rapid application development (RAD) is a method of fielding multiple design/build/test/deliver teams to work concurrently. If requirements are understood and scope is contained, RAD allows for a greatly abbreviated development timeline. This component-based approach permits incremental testing and defect repair, significantly reducing risk compared to single, comprehensive delivery. As noted, however, RAD can be costly if (1) requirements aren’t well-defined, causing a high risk of requirements defects, or (2) the design is not sound, with a minimal number of well-understood dependencies between increments, which can create a high risk of integration and maintenance issues. Figure 6-5 in Chapter 6 depicts the RAD model.

Use Lean development Techniques

Even though the project is long and complex, do not be tempted to apply more rigor than necessary. Produce documents and conduct meetings only if they add value to the project. Continually verify that the project is building the minimum viable solution. Keep in mind the motto: “Barely sufficient is enough to move forward.”

SELECTING TEAM MEMBERS AND SUSTAINING A HIGH-PERFORMING TEAM

For complex, long-duration projects, it is essential to create and sustain a high-performing team. To accomplish this, the project leader should work to:

• Select team members for the long haul

• Pay close attention to team health

• Share resources.

Chapter 10 offers additional recommendations for leading large teams on complex projects.

Select team members for the long Haul

When selecting team members for a long-duration project, keep in mind the special personality traits and coping skills that are needed. Prolonged forced interaction is simply not for everyone. For key positions, select team members who are resilient against social burnout and psychological stress.

Pay close Attention to team Health

Longer projects require different planning and development techniques to sustain momentum for the long haul; they also demand that attention be directed to the physical and emotional stresses on the project team members. Focusing on the health of the team, making strategic personnel changes at critical junctures to infuse new blood, and providing appropriate team leadership will go a long way in sustaining the team.

As a leader of complex projects, continually build your expertise in leading high-performing teams. As the project drags on and fatigue sets in, examine both team composition and team processes and make adjustments as necessary to maintain continued motivation among team members. Celebrate and reward success at key milestones rather than waiting until the end of a long project. Continually capture lessons learned about how well the team is working together and implement suggested improvements.

Share resources

On long-duration projects, critical resources may not be fully engaged. When this is the case, “lend” them out to a short-duration effort to give the team members a break, allow them to feel the gratification of completing a task or meeting an objective, and then bring them back to your project refreshed and ready to dive back in.

TABLE 9-2. Approaches for Managing Large. Long-Duration Projects

Managing Large, Long-Duration Projects
Complexities	Management Approaches
Constantly changing: - Business goals - Competitors - Global economy - Partnerships and alliances - Stakeholders - Project boundaries - Business objectives - Scope - Dependencies - Interrelationships Too large: invisible, unmanageable, unable to identify dependencies and relationships Team fatigue, leading to unpredictable human behaviors	Adaptive Adopt the appropriate project cycle and PM practices for the situation Minimize scope Delay design decision until the last responsible moment. Use incremental development Use progressive elaboration and rolling wave planning Establish a systematic estimating process using multiple estimating methods Pay close attention to team composition and health. Use lean techniques Use RAD development to increase velocity for well-understood components Conventional Perform rigorous time and cost management. Use stage-gate management Conduct rigorous risk management Use a systematic approach to develop reliable estimates

NOTES

1. The Standish Group International, Inc., “CHAOS: A Recipe for Success” (1999), 3.

2. Lara Battles, “Coping with Effects of Enforced Intimacy on Long Duration Space Flight,” Proceedings of the Founding Convention of the Mars Society (1998). Online at http://www.marssociety.com/portal/TMS_Library/MAR_98_065/library_entry_plain_abstract (accessed January 2008).

3. John A. Putman, “EEG Biofeedback and Its Potential Impact on Long Duration Space Missions,” In On to Mars: Colonizing a New World, Apogee Books (2001). Online at http://www.marssociety.com/portal/TMS_Library/Putman_2001/?searchterm=iss (accessed January 2008).

4. Robert Lane, Vincent C. Lepardo, and Graham Woodman, “How to Deal with Dynamic Complexity on Large, Long Projects.” Online at http://www.pbworld.com/library/technical_papers/pdf/32_HowtoDealwithDynamicComplexity.pdf (accessed January 2008).

5. Linda Vandergriff, “Complex Venture Acquisition,” Complexity Conference White Paper (2006).

6. The Standish Group International, Inc., “Extreme Chaos” (2001).

7. Aaron Shenhar and Dov Dvir, Reinventing Project Management: The Diamond Approach to Successful Growth and Innovation (Boston: Harvard Business School Press, 2007), 6.

8. The Standish Group International, Inc., “Extreme Chaos” (2001).

9. Ibid.

10. Mike Griffiths, “Estimation for Agile Projects” (2008). Online at http://www.gantthead.com/article.cfm?ID=240044 (accessed February 2008).

11. Jim Johnson, My Life is Failure: 100 Things You Should Know to be a Successful Project Leader (West Yarmouth, MA: The Standish Group International, 2006), 12.

12. Lev Virine and Michael Trumper, Project Decisions: The Art and Science (Vienna, VA: Management Concepts, 2008), 139-142.

13. The Standish Group International, Inc., “Extreme Chaos” (2001).

14. Ibid.

15. Jim Johnson, My Life is Failure: 100 Things You Should Know to be a Successful Project Leader (West Yarmouth, MA: The Standish Group International, 2006), 8.

16. Robert Lane, Vincent C. Lepardo, Graham Woodman, “How to Deal with Dynamic Complexity on Large, Long Projects.” Online at http://www.pbworld.com/library/technical_papers/pdf/32_HowtoDealwithDynamicComplexity.pdf (accessed January 2008), 5.