9. Attach Bolt “A” to Plank “Q”: Matching Formal Dependencies with Informal Networks

Everything today is more complex than it was yesterday. Things just keep getting bigger, better, and more complex. It’s almost like a law of nature.

This complexity makes dealing with the world a challenge. Even after people master a task, something new gets added on or a new device comes out that makes what we have obsolete, forcing us to change. We have to be constantly learning, adapting, and coordinating to make sure we’re on the right track.

Things get more complex because the knowledge that we acquire becomes embedded in the things we make. After we realize how to get two computers to wirelessly communicate, for example, that functionality becomes embedded in every computer. We build on top of that, and now that functionality is an inseparable part of that product.

Unfortunately, people still have to know how all of those fundamental parts work. Someone must understand how to make that part, what it needs to work properly, and how to make it better. Think of a computer. The major systems that make up a computer are a screen, a keyboard, a hard drive, a motherboard, and an operating system.

When you buy a computer from Dell or HP, you buy this complete package. But Dell doesn’t actually make the vast majority of the computer’s parts. Dell has gotten good at putting those parts together, but each individual part is made by a different manufacturer. There are hard drive manufacturers, operating system developers, processor manufacturers, and so on.

Taking the hard drive as an example, you can see that it’s actually made up of many subcomponents. Hard disk drives are composed of magnetic platters that are encoded by a magnetic write head with the data you want to store. The write head looks kind of like the stylus on record players. The disks spin incredibly fast, up to 14400 RPM, and the write head moves at equivalent speeds. For all of this to work you need a processor to control the write head, a motor to spin the platters at particular speeds, and the actual platters themselves. Hard drive companies don’t necessarily make any of these components themselves, but instead contract out the individual parts to third parties.

Following the chain down to the write head controller, you arrive at a computer chip company that manufactures these processors. These smaller processors are composed of a variety of circuits that create memory and calculation capabilities. These are typically all designed in house, so this is one end of the chain, right? Not yet! This company has to actually acquire raw materials, which means that they need suppliers for the silicon, copper, and other materials that they use. This normally requires the company to contract with a distributer, who will in turn have contracts with mining companies who actually dig up or chemically create the raw materials.

All of that complexity was just one path down the chain of production for a commonplace product. There are millions of these chains, which lead to networks touching millions upon millions of people.

So projects today are big. Really big. I mean really, really big. Small businesses that make furniture or craft goods from scratch still exist, but these products take months or years to produce and can only be manufactured on a small scale. The massive chains of production just discussed touch people from all parts of the world in all walks of life, and it’s clear that the drivers of the world economy are these big projects.

Building computers is relatively simple compared to some of the projects going on in the world today. These projects take place over even larger scales and have an even bigger economic impact, but they’re also prone to many different problems.

Take, for example, Boeing’s 787 Dreamliner, their airplane released in 2011. That plane has hundreds of thousands of parts. Every single one of these parts has to be designed, tested, and manufactured individually. After these parts are assembled into an individual airplane system, which also has to be designed, they have to be thoroughly tested again. When all the systems are put together to make a complete airplane, everything needs to be tested once again.

Boeing had 50 teams from 44 companies in 10 countries working to get the 787 off the ground. This feat requires a massive coordination effort, and positively dwarfs the organization of plane production only 40 years ago. In 1969 Boeing took 16 months to produce the first 747. The 787 took more than 6 years (2 years longer than originally estimated) and cost $32 billion.

The 787 is undoubtedly more complex than the 747. Its fuselage is made of cutting-edge materials, it is 20% more fuel efficient than the comparable 767,¹ and a number of technological improvements make the cabin quieter and much more hospitable than in previous airplanes.

It’s no coincidence that the incredibly complex 787 started experiencing problems only a few years after it first took to the skies. A serious battery fire, in addition to other problems, grounded 787s across the world in early 2013. Although the exact cause has yet to be revealed, the root of these problems can be traced to the complex dependencies between technological systems.

Projects have gotten more complex because our power to execute them has increased dramatically over time. The Internet, computers, ubiquitous air travel, all have radically changed what’s possible. That’s not to say that there weren’t massively complex projects before. Early examples such as the pyramids at Giza are monuments to the coordinated efforts of tens of thousands of individuals.

Previously, however, the interfaces between pieces in a project were much easier to define. When you build a pyramid, one brick goes on top of the other. Done. No changing requirements, no third party that you have to negotiate with, just put brick 1 on brick 2. Things gradually got more complex, with new technology incorporating an increasing number of innovations to become more effective.

Complexity really started to take off in the industrial revolution. At that time, companies manufactured steam engines, locomotives, and rifles on a large scale. These machines were an order of magnitude more complicated than what had come before, and required involved processes to manufacture them correctly. The concept of an assembly line developed, enabling rapid production of complex parts and machines.

Things changed by another order of magnitude in the modern era. No single person can make most of our complex devices by hand. In fact, no single person can make the devices that make our devices by hand.

This dependence on others makes coordination extremely difficult, and leaves traditional management techniques floundering.

We like to deal with complexity by making a comprehensive plan that everyone can follow. This lets us spend a lot of time upfront deciding how things should be and then executing it without the need to talk to each other. This was a fine way to do things when employees mostly worked on building cabinets or even clocks, where outside concerns were minimal and requirements didn’t change much over the course of the project.

Today, things are constantly changing, and sometimes these complex systems just don’t work the way they’re supposed to. This leads teams to change their approach as well as the parameters of their system. When they don’t communicate with the teams whose work depends on their own, bugs will invariably pop up.

Computer errors are a perfect example. Too many of us have had the experience of working on something only to be jarred out of reverie by a frozen computer. One reason computers freeze is that a single application doesn’t understand what other applications are doing on the computer. If one application has control of file A and needs access to file B, it will ask for control of that file. If at the same time a different program controls file B and needs access to file A, the computer enters a state where neither program can run. Although this example is an oversimplification of what’s actually going on, this same lack of perspective and coordination occurs in many projects today.

The jet engine project that Sosa and his colleagues studied was quite complex. The engine itself was composed of eight subsystems and had a total of 54 engine components. One team was assigned to each engine component, and six additional teams were tasked with integrating these components into the overall engine.

Although these teams all had formal plans for how they would execute this project, they hadn’t considered explicitly managing the interfaces between these different components. The researchers asked the teams to identify all of these interfaces to understand what the coordination requirements were.

In the next step researchers asked people whom they would communicate with over the course of the project. Although this is a survey-based method, researchers were asking about future interactions. If badge data had been available, they certainly could have examined who normally talks to each other to get a sense for where the bugs were going to pop up. Later sections discuss this topic in more detail, but there is definitely an opportunity to enhance this survey method with behavioral data.

Next they looked at where the gaps were. Specifically, they identified areas where many dependencies existed between teams but no planned communication was identified. These gaps represented potential problem areas and needed to be dealt with to avoid any problems with development, but their causes are complex.

Organizational boundaries are a major source of collaborative discord and the gaps discussed above. Almost by definition, if you don’t report to someone, then you don’t have to communicate with him. Unless you go out of your way to communicate with him to talk about a dependency, a gap will result. Although formal structures are a blunt instrument for enforcing collaboration across boundaries, because your salary depends on effective communication with your bosses, you can bet that you’ll make the effort to talk with them.

Of course, speaking with someone is difficult if you’re not in the same place. This is often the case with teams working on complex projects, because if a team is composed of hundreds of people, then co-locating them is often difficult. Even if teams are on the same campus, they are normally on separate floors or in separate buildings. As discussed in Chapter 4, this setup makes it unlikely that they will ever talk to each other.

The boundary problem can be attacked in a few ways. Creating new reporting relationships is certainly one way to go, and this approach is favored in many organizations.

Assigning a team to specifically coordinate interfaces between teams can also be effective. This is almost like a light form of matrixing, where some formal processes are in place for reporting to these integration teams. Using these teams might be preferable to modifying the org chart, however, because they are explicitly formed to coordinate and facilitate communication rather than to order around.

For these coordination teams to be effective, they need to be able to easily communicate with all teams involved, and that means they need face-to-face communication. For issues as complex as jet engine construction or computer chip design, rich communication channels are needed to keep everyone on the same page. Whether or not a company uses coordination teams, getting people in the same place at least part of the time is important.

As discussed in Chapter 4, research shows that the more time teams spend together in person, the better they perform. However, keeping the amount of travel reasonable is important. There are simply diminishing returns as these different teams meet more frequently, because eventually people will be spending most of their time traveling. They won’t have time to get anything done. Still, the $25,000 spent on a few dozen plane tickets saves millions of dollars on extra labor and time lost. Truly a no-brainer.

These priorities matter not just for financial considerations, but also for how communication time is allocated. Although the tendency is to focus most of our time concentrating on the hard problems, the interfaces that require a lot of coordination, too often the small stuff slips through the cracks. When Sosa and his colleagues looked at how people wanted to spend their time, they typically ignored communicating with teams who were working on non-critical interfaces.

Problems arise when these high-priority problems become employees’ sole focus. By not spending a little time touching base with people on routine interfaces, you can bet that those interfaces will soon become far from routine if something changes.

A good analogy is how the Massachusetts Bay Transportation Authority (MBTA) deals with the crumbling state of its subway system. When it was first built in the late 1800s, the subway system was a technological marvel, the first active subway in the United States. The system has expanded over time to cover most of the greater Boston metro area, and is the main mode of transportation for about 30% of Boston’s commuters.

Over time the subway system fell into disrepair. With a massive load of debt hoisted on it by the state, the MBTA was faced with some harsh budgetary realities. It could fix critical problems and ignore the less-urgent ones, or it could take on a small amount of additional debt and fix the non-critical problems as well. It opted to focus solely on critical problems, and the result has been financial catastrophe. The MBTA has been forced to take loans against future revenue streams to pay for its yearly upkeep, and it is constantly on the precipice of going bankrupt. The situation is so bad that the MBTA actually employs blacksmiths to forge train parts that aren’t made anymore.³

The issue with the MBTA’s approach is that when you ignore one of these small problems for a few years, all of a sudden it becomes a critical problem. These problems then cost far more to fix. Because the MBTA does not have the funds to adequately deal with its problems, it falls further into debt. This is many times the debt it would have incurred if it had taken the up-front cost of fixing these smaller problems, but the focus on only critical issues obscured the long-term objective.

The general problem with how people prioritize in big projects is the tendency to focus on the big things and ignore the small stuff until they become big things. This is precisely where informal communication is critical. These small things aren’t going to come up in formal meetings because they’re not important enough to take everyone’s time. But they are important enough for a chat around the water cooler, and communicating information about these smaller issues will enable the company to head off future problems well before they occur.

This again indicates the importance of enabling informal communication between teams, whether it be through cultural or workplace interventions or communication tools that facilitate discussion. In general, however, the communication tools available today are not very good at this. Internal instant messaging programs and social networking services such as Yammer, which is like an internal Twitter for companies, rely on high employee adoption and a willingness to connect with people with whom a person doesn’t normally communicate. That’s hard to achieve, and a challenge most of these technologies don’t directly address.

This communication is also critical for checking our common assumptions, which are another major cause of coordination headaches. We often make assumptions about how to proceed on a project based on what we’ve done in the past, assuming that everyone is on the same page. In projects that span millions of individuals with many different backgrounds who work on hundreds of different projects at the same time, this assumption is more often than not incorrect.

These assumptions can also pop up when people rely too heavily on standards. At the start of a project, team leaders will spend time planning and designing the overall structure of the final product. Eventually these specifications will get more and more precise, until they describe in detailed language how each component interface is expected to behave.

The problem is that components often don’t behave as expected. Unforeseen difficulties can force a team to modify component specifications, and strategic changes from the top can leave teams struggling to adapt their plans. Whenever these changes happen, the potential for error is introduced. While others are continuing their work and assuming that the old specifications still apply, other teams have completely changed their expectations and are working down another path.

As Chapter 3 touched on, one of the major benefits of a cohesive face-to-face network is the ability to build a common language and a common set of assumptions. In these larger projects, scale prevents that from happening. Having a cohesive group of 300 people, let alone one million people, is just not possible. This situation, however, indicates a huge opportunity for incorporating badge data into the larger organizational process, particularly when companies can pair this data with information from databases on dependencies in software and engineering systems.

Beyond the sheer scale of these programs, the way that they’re constructed also makes coordination problems a very real possibility. At a very simplified level, a computer program is essentially a collection of modules that interact with each other. These modules can be large programs in their own right or small routines that perform the most rudimentary tasks.

Consider a simple program with two modules. One module raises a number to a power (call this POW), and the other module asks the user to input two numbers (let’s call this INPUT). POW expects to receive two numbers, x and y, and then returns x^y. Suppose INPUT, however, thinks that POW will actually return y^x. This would most likely cause INPUT to completely fail or make substantial errors when it uses POW, all because two modules that depended on each other didn’t know how to interact.

This example is extremely simplistic, and a straightforward problem such as this would likely be resolved quickly. In real software, however, the problems are much more nuanced, depending on the overall state of the program as well as the computer as a whole.

Given their importance, dependencies have been put center stage in the software development community. Dependency tracking tools have been incorporated into nearly all major development environments, which programmers use to create software, and for decades methods for automatically detecting dependency issues have been a hot topic in computer science.

More recently, the importance of communication has entered into this equation. Researchers and practitioners alike realized that using formal reporting tools alone didn’t solve the need for interaction between programmers to ensure that dependencies are adequately covered. They even developed a terminology for these issues.

Dependencies can either be adequately covered by communication, or they can fall by the wayside. Congruence occurs when there is alignment between software requirements and communication patterns—that is, when there is communication between the programmers responsible for dependent code modules. In contrast, gaps arise when there is no communication between these programmers. Graphically, the relationships look like that shown in Figure 9.1.

Figure 9.1. Programmers, code, and dependencies

In Figure 9.1, P₁ and P₂ represent programmers, and the gray circles represent different code modules. Lines represent dependencies/responsibilities for modules. In this case, because module X depends on module Y, P₁ is the dependent and P₂ is the parent.

As you can probably infer from the earlier discussion, gaps are overwhelmingly associated with software bugs. In a landmark study, Marcelo Cataldo and his colleagues showed that congruent relationships resulted in 32% faster completion times as compared to other relationships.⁴ When development cycles can stretch on for years, this represents a significant benefit of congruence.

Communication, however, is not necessarily a normal part of the development process. In traditional software development, dependencies and modules are assigned in the planning stages of a project, and execution proceeds assuming that programmers will faithfully reproduce this master plan. This usually doesn’t happen, however, because anticipating problems that can appear during development is nearly impossible for planners. Whether the changes arise from unrealistic response time requirements, aesthetic changes that occur mid-stream, or something else, they all require coordination and communication.

Lately, development environments have incorporated communication functionality into the overall program. This often takes the form of comment fields and instant messaging channels that developers can associate with particular pieces of code.

These tools have become even more crucial to use as developers are increasingly spread over larger and larger distances. Due to its perceived formality and the availability of cheap labor with equivalent skill sets in developing countries, software development is probably the most distributed white-collar profession.

Unfortunately, development across multiple locations has been associated with project delays and increased work for individual employees. This mostly results from the lack of face-to-face communication and differences in time zone, which makes having rich interactions with other people on the team difficult. These communication effects would, presumably, affect the prevalence of congruence and gaps.

Along with Kate Ehrlich and Mary Helander from IBM, I investigated precisely how co-located and distributed teams collaborate and how this communication related to gaps.⁵ By collecting a rich data set not just on communication, but on the dependencies between different code modules, we were able to discover a number of ways that both co-located and distributed teams could improve their coordination.

In this project we examined a medium-sized development team of 161 programmers working across 20 sub-teams. Some people were on multiple teams, and some of the teams were co-located, whereas others spanned multiple locations. Importantly, all team members used English as their primary language.

These teams heavily utilized a software development environment that recorded data on code dependencies as well as provided communication tools for people to comment on pieces of code or directly with each other. We scraped this data from the development environment, in total obtaining records on thousands of dependencies and tens of thousands of comments.

Overall, the average programmer had code that depended on 32.5 other modules in the project, with a high of 177 dependencies. Even the average number of dependencies is a lot to keep track of, especially because each of these dependencies can change at any time. While not all of these dependencies are crucial, it often falls on the dependent to make sure that none of his interfaces with other modules has changed. This makes sense, because someone working on a critical module would have a hard time satisfying the requirements for the 100 or so modules that depend on it. However, it does mean that dependents need to keep on their toes.

A look at the breakdown of congruency and gaps revealed what we might expect. Gaps were much more common when dependencies spanned groups in different locations. In general much of the work in this group was done remotely, so there were actually 22% more dependencies in remote teams. Figure 9.2 illustrates the results.

Figure 9.2. Congruencies and gaps by programmer location

For co-located groups, about 55% of all dependencies were congruent, whereas there were gaps for 45% of dependencies. Remote groups essentially flipped these numbers, with 47% congruencies and 53% gaps. This is an extremely large difference. These results imply that remote programming groups are 8% less effective than co-located groups. Also remember that in this case everyone’s native language was English. What would have happened if some of the groups had difficulty communicating in the lingua franca of the other groups?

To calculate the presence or absence of a gap, however, the study only examined whether there was any communication between the two parties relating to the dependency. As you might imagine, a single comment probably indicates a low level of coordination, whereas a high number of comments indicates that people are actively engaged in addressing potential issues. Because this is all electronic communication, nothing is physically preventing people from communicating. However, the numbers are pretty overwhelming.

The average dependency for remote groups received 8.4 comments. Co-located dependencies, on the other hand, had an average of 38.1 comments. In this team, face-to-face communication clearly bolsters the overall level of collaboration between coworkers. This enables them to not only address dependencies with more regularity than their remote colleagues, but also to address them more effectively.

As these projects grow in scale, having everyone working on them in the same place can become infeasible. As the study results show, this is potentially a major issue. Teams working remotely need to place an even greater emphasis on achieving congruence, and using lightweight digital communication tools alone might not be enough.

Collaborators need to actively set up video conferences, phone calls, and chat sessions to make sure that these dependencies are being addressed. Although this extra communication takes time out of the workday, spending an extra 5% of one’s time talking with collaborators versus an extra 30% of one’s time debugging seems clear-cut.

Organizations can also work to assign “brokers” who will actively try to connect distributed groups. These brokers are ideally people who travel frequently to different locations or are at least in close contact with relevant stakeholders. Their informal sense of who should be talking to whom and making the act of connection a part of their daily routine will pay huge dividends for the organization in the medium and long term. These introductions create congruence, which speeds development time for a particular project and creates relationships that span distance and organizational boundaries. As the examples in this book have illustrated, those relationships are the glue that holds companies together.

A last important note is that dependencies need to be matched by the physical layout of the workplace. Chapter 4 covered how distance is an important driver of communication; that result emerged at a different level in this project, which showed in striking relief the added importance of dependencies. By mashing these two findings together, companies can accurately choose an ideal workplace layout.

When companies decide which teams sit where and even what cities teams are located in, dependencies need to be a major consideration. Although at the outset of a project knowing what dependencies will emerge and who will be responsible for which modules might be difficult, the general structure should be relatively clear. This allows companies to strategically choose team locations to drastically reduce the prevalence of gaps using the power of serendipity. In this case bumping into other people in the hallway or at the coffee area has an immediate impact on performance by helping people coordinate their work, to speak nothing of the long-term benefits of these interactions.

I don’t want to imply that teams need to constantly move desks or transfer to different cities as their short-term dependencies change. That would be a logistical nightmare and end up costing an inordinate amount of time and peace of mind for employees. The major point is that organizational strategy at the macro level and dependencies at the micro level need to be manifested in the design of the workplace. Companies must strive to make this critical face-to-face communication as likely as possible, because as the study results emphasized, digital tools can’t be relied on to bridge the gap.

If companies’ collaboration methods don’t change, projects such as the Dreamliner are going to get even longer and more expensive. In about 40 years Boeing went from making a new plane in 16 months to 120 months. Even at a much slower rate of change, in another 40 years it would take a company such as Boeing well over a decade and cost hundreds of billions of dollars to develop a new aircraft. Something has to change. The primary time sink in these projects is the need for congruency. Making changes such as the ones suggested in this chapter can effectively reverse this trend. That’s not to say that Boeing’s next aircraft will take 16 months to produce, but it could.