Complexity Rising

Complexity, normally referred to as the state of something that has many interconnected parts and interrelationships, is part of our reality. In many aspects of life, humans have a tendency to push the norm or current status quo. Our ever-increasing wants and desires drive our collective environment toward more challenging and exciting ends. This is true in our relationships, activities, careers, and especially in the products, services, and other capabilities we utilize.²

Increasingly, we as consumers are demanding more complex solutions driven by specialization and customization demands to meet our individual needs and desires. This complexity manifests itself in the following ways: designs have become more complex as the desire for more integrated features increases; the development of solutions has become more complex by the distribution of resources across the planet; and new innovation has to be married with desirable user experiences.

This push for more exciting solutions has moved us to a world of more connectedness, which is increasingly leading to the pervasive rise in complexity. Things that were once independent (think of phones, cameras, and computers) are no longer so, and the neat and simple construct of the past is no longer viable.³ For people like Richard Cook who work in industries in which complexity has always been central to the way they do business, this is not a new phenomenon. For many others, this rise in complexity has emerged as a primary challenge to the way we have historically operated within our businesses and organizations.

Recently, the Economist Intelligence Unit (EIU)—an independent business within the Economist Group that provides intelligence and insights on business operations and markets—conducted a survey among business leaders to ascertain the challenge of complexity.⁴ Three hundred executives were surveyed and the conclusions were as follows:

• Doing business has become increasingly complex.
• The biggest cause of complexity is customer expectations.
• Businesses are finding it increasingly difficult to cope with this rise in complexity.
• Historic business processes and organizational structures are adding to the complexity challenge.

As noted by the EIU research, one of the major causes of rising complexity is the result of customer expectations, and our historic ways of doing business have become insufficient to manage the new levels of complexity.

While speaking at a conference on the topic of complexity and systems thinking, we were approached afterward by a gentleman who seemed to relate to what we were espousing. “This is exactly what we need,” he stated. “We are being consumed by complexity.” What he went on to describe was a situation where the infrastructure technology capabilities that were being developed and deployed within his organization (a well-known banking institution) had become so complex that they could no longer be effectively managed as a single project.

He explained that the firm attempted a multi-project approach for the development of their capabilities by sub-dividing the effort into multiple projects, each with a dedicated project manager and team. What the gentleman saw in our presentation that made him realize that this approach was insufficient was the large network of cross-project interdependencies and interconnectedness that existed between the projects. These interdependencies were being left unmanaged. The following explores this in more detail.

The interdependencies between projects consist of a series of cross-project deliverables, coordinated tasks, cross-discipline decisions, and shared management of risks and problems that are encountered on any complex work effort.

As the term implies, interdependent projects are those that have a dependence upon the delivery of an output from one project to another. Project interdependencies normally equate to deliverables, tangible outputs from one project team that are delivered to one or more other project teams. Interdependency between the project teams is formed when a deliverable from one project team is needed to successfully complete the work of a second (or more) project team(s).

To illustrate how interdependencies between projects are a determining factor in complexity, Figure 4.1 provides a simple example consisting of four projects.

The three deliverables in Figure 4.1 are: 1) the computer-aided design (CAD) files delivered from the hardware project team to the manufacturing project team, 2) the control software delivered by the software project team to the manufacturing project team, and 3) the manufactured product delivered from the manufacturing project team to the system test project team.

This series of deliverables creates a highly interdependent relationship between the projects. For the first deliverable, the manufacturing project team needs the CAD files from the hardware project team for the production of the physical circuit boards. An interdependency is established between the hardware development and manufacturing project teams.

The manufacturing team cannot build a circuit board without the control software from the software project team that is needed to power up and control the board. This, of course, would lead to failure of the system test project team to complete the test of the product.

The three deliverables described create a highly interdependent relationship between the four project teams. These interdependencies between project teams need to be managed as closely as the deliverables created by each project team. Focusing on completion of deliverables alone is not enough to be successful, especially on projects with significant complexity and numerous interdependencies.

Determining the Level of Complexity

The level of complexity is highly dependent upon the number of cross-project interdependencies and should be a key deciding factor of the management approach used to oversee the initiative. To illustrate, we will simply add two elements to the example above. First, we will assume that the product requires the development of an integrated circuit (IC), and second, we will assume that the product requires the development of an enclosure to power, cool, and contain the other components. Our expanded set of projects and dependencies is illustrated in Figure 4.2, with each arrow representing a potential interdependent deliverable between the projects.

One can visualize that the complexity of the interdependency structure between the projects increases with the addition of just two elements. In fact, a total of 30 potential interdependencies between the 6 projects are now possible (see “How Many Dependencies Are There among Five Projects?”). One can quickly see that as the magnitude of the effort increases, the number of interdependencies among project teams becomes a management challenge, thus highlighting the need for an effective approach to provide focus on the interdependencies between projects.

One could make the case that the project managers from the four projects illustrated in Figure 4.1 could adequately manage the work effort described on a cooperative basis without the need for a program manager. We do not disagree that this may be true. In fact, projects of this nature are managed effectively every day by capable project managers. However, industry benchmarking, research, and personal experience show us that this approach is sustainable only to fairly low levels of project complexity.

Four serious problems are consistently encountered when complexity overrides traditional project approaches. First, as complexity rises, the project managers are required to spend the majority of their time planning, executing, and delivering their pieces of the work effort. Eventually, the work output has to be integrated, which is rarely a flawless event. The project can experience significant schedule and cost overruns due to rework and extended integration efforts caused by lack of focus on interdependencies.

A second problem is encountered when schedule pressures exist due to internal or external factors. When schedule pressure exists, the natural tendency for project managers is to hunker down and execute their project deliverables, with little or no time allocated to managing the business goals driving the need for the project. Business requirements such as return on investment or cost reduction goals may be left unattended. In this scenario, the capability may achieve all technical features and functional requirements intended and may be delivered to the aggressive schedule target, but the project may also be deemed a failure because the business goals were not achieved.

Problem three is that someone needs to be responsible for delivering the whole solution to the customers and stakeholders. As the scenarios in the first two problems begin to unfold, the project teams begin focusing on their respective elements of the solution exclusively; after all, that is their “job one.” Therefore, the technical aspects of each functional discipline take precedence over the integrated solution when, in fact, just the opposite has to occur to adequately meet customers' expectations.

The fourth problem that can be encountered is that interdependencies between projects can grow at a rapid rate as demonstrated previously. Eventually, the cooperative approach by the project managers to manage the mesh of interdependencies breaks down, and no one is left to manage the large number of interdependencies between the projects. This is a critical point. There are two vectors of complexity that need to be managed. The first vector is the definition of responsibilities and deliverables for each individual project manager and his or her project team. The second vector of complexity involves defining, sorting out, and managing the large number of interdependent tasks, responsibilities, and deliverables that require joint sharing of information, planning, and executing between the project managers and members of the various project teams.

This nature of interrelationship and collaboration necessary today is ubiquitous and has caused us to shift our focus from the management of individual elements alone, to also focus on a more macro-level approach of managing the interactions among the elements. This is the basis of systems thinking.⁶

Systems Thinking

For the most part, we see the world as increasingly complex and uncertain because we are trying to use inadequate concepts to explain it. In the world of minimal interconnectedness of the past, we were able to employ basic analysis to explain things. First, we isolate the parts we are trying to understand; we then characterize the behavior of each individual element we are studying; and finally we aggregate our understanding of the parts to explain the behavior of the whole.

In the current world of interconnectedness, however, a different way of thinking is required to understand and characterize behavior. Systems thinking does not require separation of the parts from the whole. Rather it respects the interrelationships between the parts. More importantly, it focuses on the interactions between the parts.

The systems thinking approach allows one to step above the fray of details associated with complexity to view the world from a simpler vantage point. A recent article in the Wall Street Journal recounted Henry David Thereau's maxim for “simplicity, simplicity, simplicity.”⁷ Thoreau was convinced that our lives are “frittered away by details.”⁸ What he was telling us was that complexity has congested our lives and that the solution to complexity is “simplicity.”

Fortunately, people are continuing to come to the understanding that systems thinking is an effective approach to simplification of complexity and uncertainty. George Reed, a retired army colonel, is one of those people. Regarding the need for systems thinking, he noted: “Leaders operate in the realm of bewildering uncertainty and staggering complexity,”⁹ and to successfully operate in any contemporary market or organization today requires systems thinking.

Systems

Ludwig von Bertalanffy, an Austrian-born biologist, is recognized as the father of general systems theory and systems thinking. Leveraging von Bertalanffy's work from the 1940s and '50s, a system can be thought of as a combination of interrelated parts that collectively work together as an integrated whole to achieve a common purpose.¹⁰ Each part by itself is of little or no value. Value is only created from the interaction with the other parts of the system to perform the function of the whole. Think of an analog clock. The parts of a clock, such as cogs, dials, and hands, are of little value by themselves. They only provide synergistic value (customer determined value) when they are assembled together and collectively perform the function of the clock (the system).

A system can be viewed as a simple entity consisting of four primary elements: 1) inputs, 2) outputs, 3) interdependent subsystems, and 4) an environment within which the system operates. Figure 4.3 illustrates a simple model of a system.

As we shared in Chapter 1, the first documented case of the use of program management in the U.S. dates back to 1957. Within that description also lies an example of a complex system being described by a simple model. The system described was an undersea weapons delivery system (remember, this was the era of the Cold War). Figure 4.4 illustrates the model showing the major components (or subsystems) that comprised the overall system.

The diagram also shows the network of interdependencies between the subsystems. We have greatly simplified this interdependency network for illustration purposes. In reality there were hundreds of interdependencies among the subsystems.

We use this simplified example to make a number of key points: First, complexity in capability development has existed for a long time. Second, systems thinking has successfully been employed as a way to simplify capability complexity. Third, there are a number of key characteristics of a system that are relevant to the way we should manage complex programs today. These characteristics include the following:

Organization: A system is organized in a way that brings structure and order to the components and to the whole. It is the arrangement of the components that helps the system accomplish the overriding purpose or goal.

Interaction: Interaction refers to the manner in which each of the components of a system functions with the other components. It is the interaction between the components that forms the interrelationships within a system.

Interdependence: The interaction between the components of a system forms a network of interdependence where the function of one component is dependent upon the function of one or more of the other components. For example, the output of one component is required as an input to another component in order for it to function properly.

Integration: Integration involves the aggregation of the system components into a single entity, which can achieve the overarching purpose. The functionality of a system can only be accomplished when the activities associated with each of the components are tied together.

Our intent here is not to provide a crash course on systems theory. That is beyond the scope of this book. We provide the information above, however, to make the point that there is great value in taking a top-down, systematic approach to define your next program, especially as the complexity of your program increases.

This approach applies for strategy-led programs and formulated programs, as well as for mandatory programs that are intended to meet regulatory or compliance stipulations. By taking a top-down approach, a program manager will be working to create a critical component of their program—the program vision.

When working with organizations and their program managers, we use the whole solution concept to establish the program vision. You will likely now recognize the value of systems thinking in the discussion that follows.

The Whole Solution Concept

With the exception of a small percentage of companies, most organizations do not and will not utilize a full systems approach to create new capabilities. This does not mean that they cannot benefit from the concepts of systems thinking, particularly if they wish to implement some of the fundamental aspects of program management, such as the importance of taking a top-down approach where one begins with a firm's business goals and the strategies to achieve them as the hinge points for defining and managing a program.

When complex, system-level opportunities arise, a full range of company specialists are needed to create solutions. We have found it useful and effective to use the whole solution concept when first introducing systems concepts to an organization to establish the program vision, or the capability end state, that the program is intending to reach.

The concept of the whole solution is not new. It originated from the marketing discipline when Geoffrey A. Moore coined the term “The Whole Product” in his book Crossing the Chasm. He defined the whole product as the products and services that best meet the customer's wants and needs (see “The Concept of the Whole Product”).¹¹

If we shift from a marketing focus to a capability delivery focus, the whole solution can be defined as “the integrated solution that fulfills the customers' expectation.”¹² In other words, the defined solution must holistically meet all of the customer's expectations.

As many companies and organizations have learned, meeting the customer's expectations becomes the means to achieve the strategic business goals of the firm. If customers put a priority on receiving the whole solution, the concept then needs to be part of a company's business strategy. Once it is part of the business strategy, the program management discipline becomes responsible for developing and delivering the whole solution.

If we purchase a laptop computer, for example, we wouldn't consider it acceptable if we were delivered a box of circuit boards, a second box that contains the enclosure, another box containing peripheral devices such as memory and network adapters, and finally an envelope containing the computer software applications and operating system on a set of compact disks. Rather, unless we're a computer hobbyist or a systems integrator, we expect to receive an integrated laptop that we can unpack, plug in, and begin using with the applications we most desire. Thus, we want to experience the delivery of the whole solution.

It is helpful to think of the whole solution consisting of two parts: 1) the core components, and 2) the enabling components. The core components are the tangible elements of the whole solution that, when integrated, constitute the physical capability developed. In systems language, these are the subsystems of the integrated system. The enabling components are the additional elements needed to ensure the capability meets customers' expectations.

Let's look at an example to illustrate. Imagine that you are in charge of leading the program commissioned to create the next generation smart phone for a leading phone manufacturer. Your whole solution diagram may look something like the one illustrated in Figure 4.5.

The whole solution begins with the core components that consist of the physical elements that make up the phone such as the digital circuitry, the embedded software, the radio device, and the enclosure packaging (keep in mind this is a simplistic view for discussion's sake).

The whole solution also includes other important elements needed, such as a software application development platform, interface to the wireless communication infrastructure, manufacturing of the product, quality assurance, and customer support for the users of the whole solution. These are the enabling components of the whole solution that are needed to ensure complete customer and user satisfaction when the capability is delivered.

In Chapter 1 we offered a model that shows how program management links execution to strategy by integrating the work flow and deliverables of multiple interdependent projects to develop and deliver an integrated solution. What we didn't discuss at that time was the importance of the whole solution in providing a program vision that focuses the output of the interdependent projects toward a common solution that will enable the achievement of the anticipated business goals. Figure 4.6 illustrates how the whole solution fits within the strategy/execution alignment model.

System design work requires deep collaboration across the functional specialties of a company. By creating the whole solution diagram, a cornerstone element of the program vision is established that provides a visual representation of the integrated solution that will meet the customers' expectations. The whole solution helps to ensure that a holistic view of what it will take to meet those expectations is a driving force in the work of the program team. A strong program vision is in place when each of the members of the program team can see how their work efforts contribute to the creation and delivery of the integrated solution.

The Program Architecture

The term architecture refers to the conceptual structure and logical organization of a system. It includes the elements of the system and the relationships among them.¹³ A program architecture is therefore the conceptual structure and logical organization of a program. It is composed of the constituent projects of the program as well as the enabling functional components required to create and deliver the whole solution.

To create an effective program architecture, it is helpful to again take a systems approach. In any case where a new capability will be created and introduced into the environment, the program serves as the delivery mechanism for that capability and should be designed and structured in a systematic fashion.

Why is this? If a program is charged with delivering an integrated, whole solution, it has to be architected in a manner in which the whole solution is effectively created. The whole solution will inform one of the primary elements needed of the program architecture. Take for example the simplified whole solution diagram for the creation of a new capability such as a new cell phone, as illustrated in Figure 4.5. The program architecture is easily derived directly from the whole solution, with additional thought toward which of the components needs to be structured as a project, and which components are stand-alone work efforts conducted by members of functional organizations. Figure 4.7 demonstrates an example of how the program architecture might be designed for the cell phone solution.

In this example, all of the core components will be structured as projects, each of which will be led by a dedicated project manager. Additionally, two of the enabling components, the application platform and manufacturing, are efforts that require them to be structured as projects within the program. The other components are not structured as projects, but dedicated personnel from the appropriate functional organizations (or external organizations if the work is out-sourced) must be included as part of the program team.

As we stated in Chapter 1, we like the fact that PMI's program definition includes the following statement: “Programs may include elements of related work outside of the scope of the discreet projects in the program.”¹⁴ If one utilizes the whole solution concept to help define the architecture of a program, the elements of a program that are not managed as a true project, but are essential to meeting the expectations of one's customers, emerge.

This systematic approach to architecting a program works well for strategy-focused and compliance-type programs. But what about for formulated programs? Through our experience, we have found that the process works equally well, but the process for creating the program architecture is not as fluid as it is for the other types of programs.

With formulated programs, one begins with a suite of projects that one wishes to combine into a single program. The obvious approach is to create the program architecture from the existing projects. This is not our recommended approach, however. It is best if one starts by identifying the business goals desired, then develop a capability definition that will achieve the business goals (Chapter 6), and then define the whole solution needed to successfully achieve the goals. At that point, the existing projects can be evaluated to determine their relevance to creating the capability and delivering the whole solution, and therefore whether they should be included in the program architecture. Changes to the suite of projects and the charter of each project can then be made as appropriate.

Adding Dimension to the Program Architecture

When utilizing program management to create and deliver the whole solution, it is helpful to view management responsibilities within the program in two dimensions: vertically and horizontally. This concept, as illustrated in Figure 4.8, is a core characteristic of program management. The figure shows a simple example of five components of the cell phone example discussed previously.

Both the vertical and horizontal elements involved in the management of a program are evident in Figure 4.8. First, let's look at the vertical dimension. The work of each function or organization involved in the program is structured as a project and is led by a project manager. These project teams of specialists are responsible for the development and delivery of their respective pieces of the solution. The end result or output of the work accomplished by each project team is known as the project deliverables or outcomes. As an example, the circuit development project team is responsible for delivery of the various circuit boards that make up the whole solution; the software project team is responsible for delivery of the software stack (application, drivers, and utilities); and so on for each of the project teams. Again, the project managers are managing vertically, or within their specialty domain, to develop their respective element of the whole solution and deliver it to the other members of the program team.

The horizontal dimension of the program is represented by program management, which cuts across the project teams and synchronizes and integrates the work flow and outcomes of all constituent projects.

Using systems design to provide an integrated solution prevents the vertical elements of a program from taking a silo approach to developing and delivering their component of the whole solution. Rather, it establishes working relationships between the teams of project specialists. To be successful, each project team within the program is highly dependent upon cross-project deliverables from the other project teams. In Figure 4.8, the cross-project deliverables are represented by the horizontal lines between the projects. We call this cross-project interdependency the “space” between the project teams. Management of the space between the projects is the responsibility of the program manager and involves the identification and synchronized delivery of the project interdependencies. This is accomplished through interface definition, cross-project coordination, communication, decision-making, and problem solving.

When we consider the two dimensions of a program, it becomes evident that the successful management of that program requires team effort. Establishing the right team and creating an environment of effective cross-organizational collaboration is a critical aspect of creating and delivering the whole solution. This brings us to the next chapter where we discuss building the appropriate team and creating an environment of collaboration.