29

The Research Challenge

In his provocative book The Halo Effect,¹ Phil Rosenzweig lists nine “delusions”—errors in logic or judgment—that can lead to a distorted understanding of organizational performance. In my experience, three of these are particularly problematic.

Was It Just Luck?

The first barrier that gets in the way of drawing conclusions concerns the role of luck. We all want to learn from success, so we look at the victors and draw conclusions. We do it with organizations, with successful executives, and with winners of ocean races. We all want to be winners. But a lingering question concerns the role of luck in winning performance.

Michael Raynor and his colleagues with Deloitte Research use an interesting example to make the point. They describe how a professor at MIT, Rebecca Henderson, illustrates the role of luck to her class in strategic management.²

At the beginning of the class, she asks all the students to stand up and flip a coin. After the coin toss, only those whose coins came up heads are told to remain standing. After six or seven rounds, only one student is still standing—the winner!

Professor Henderson runs up to congratulate the successful student, offering to write a case study about him or her or do an interview in Fortune magazine. The student is a hero, a winner. But in this instance, of course, it's clear that there was nothing about the student that contributed to his or her success. The student benefited from what we would call luck.

This theme has been echoed by others. In Fooled by Randomness, Nassim Taleb³ argues that we often mistake luck for skill. This tendency occurs in all areas of life but is especially apparent in the world of markets. Frequently, according to Taleb, we mistakenly believe that a particular investment strategy works, an entrepreneur has vision, or a trader is talented when, in fact, the successful performance is almost entirely attributable to chance alone.

I understand the problem, and I know that there is a degree of luck involved in any successful endeavor. But when I look at boats that successfully complete the Sydney to Hobart Race—and particularly those that take home the coveted Tattersall's Cup awarded to the overall winner of the race—I am confident that there is much more than luck involved. I've tracked the performance of the Midnight Rambler for more than a decade after the 1998 race, and her success involves much more than flipping a coin.

To win the Sydney to Hobart Race in a hurricane, the team would have to flip those coins for 628 miles in Force 12 Hurricane conditions. And they would have to keep winning the coin toss for a decade. In any particular race, of course, there is an element of chance, and other boats were close competitors of the Rambler. But it is this sustained record of success in different racing conditions and different time periods that gives me confidence that they are doing much more than capitalizing on chance.

Correlation vs. Causality

In a classic experiment, renowned behavioral psychologist B.F. Skinner designed a box to drop food pellets to hungry pigeons. The food was presented at regular intervals, and the reward had nothing at all to do with the behavior of the bird.⁴

What Skinner found was that the birds became conditioned to perform a specific response that—at least from the perspective of the pigeons—was connected to what the birds happened to be doing at the time they were rewarded. Skinner's observations were very telling and somewhat comical. One bird was conditioned to turn counterclockwise around the cage. Another repeatedly put its head into one of the upper corners of the cage. A third developed a “tossing” response. Two birds developed a pendulum motion of the head and body. And yet another bird made incomplete pecking movements directed toward—but not touching—the floor.

Apparently, Skinner's birds tended to repeat whatever they were doing when the food hopper appeared. In a sense, what Skinner had discovered was delusional or superstitious behavior on the part of the birds. They behaved as if there were a cause-and-effect relationship between a particular response and food, when, in fact, there was none.

To really understand organizational performance—in this case, effective teamwork—we need to make sure that the elements we identify as ingredients really are causes and not just accompaniments. We need to tell a coherent story, and we need to be able to separate the causes from the effects.

Knowing What to Look For

The third challenge in examining the characteristics of effective teams is sorting out all the different factors that can contribute to high performance. Suppose you find the team that you're confident is good and not just lucky. How do you know what the really important ingredients are—the things that make a difference? Here's an example of the problem.

A Wall Street Journal article described a twelve-year study of 361,000 middle-aged American men.⁵ The research showed that among non-smokers there were 1.09 suicides per 100,000 person-years. The rate increased steadily with the number of cigarettes smoked, reaching 3.78 suicides per 100,000 person-years for people who smoked three packs of cigarettes a day. And there was more.

Not only did the risk of suicide increase steadily with the number of cigarettes smoked, but the relative risk of murder also rose with smoking. Two-pack-a-day smokers were twice as likely to be murdered as nonsmokers.

It was hard to conceive of a way that smoking could cause people to commit suicide, and it was even harder to understand why murderers would be intent on hunting down smokers—preferring them to nonsmoking victims. This appeared to be another case of confusion between cause and effect. It seems much more likely that smoking, suicide, and murder were all part of a much bigger picture. All three increased together, but the research gave no hint about the underlying cause of all three.

If the researchers in the study had spent time living in areas with a high concentration of smokers who were likely to be murdered or commit suicide, they might have reached plausible explanations for the bigger picture of underlying causes. Lacking that firsthand experience, there was little that could be gleaned from statistics alone.

The problem of knowing very specifically what causes what has plagued books about organizational performance. When people are asked about the things that contribute to success, their memories are often biased and incomplete. It's hard to be objective, and we tend to remember the things we want to remember. And if we try to understand underlying causes based on analyzing more “objective” information—for example, financial statements, newspaper articles, and so forth—there is another problem. How do we know that—as distant observers—we're not identifying things that may be related but are relatively unimportant?

My Approach

The reality is that there is no feasible scientific method that will completely resolve all of these problems. If I were a medical researcher, I could randomly assign some patients to receive a dosage of the special teamwork formula—that is, the strategies I believe are important. I could use a control group of teams that employs strategies I don't think are important. I could even assign another group of teams to get a dosage of other randomly generated strategies. I would then see if my teams did better in the experiment, and I would confirm my results within a certain range of probabilities.

I can do laboratory experiments within certain limited situations to learn more about teams. But my ability to understand the dynamics and successes of teams that are really at The Edge is limited. Nonetheless, even if “scientific truth” about successful teamwork is elusive, it is possible to gather significant insights about teams. It is possible to draw reasonable conclusions that will help people and organizations struggling with significant problems.

My approach is based on two primary strategies. First, in an effort to rule out success being the result of luck—of chance alone—I looked at sustained performance over a period of years. Second, I reach my conclusions by getting firsthand experience with the research topic.

I immersed myself in the world of teamwork—particularly teamwork in ocean racing—and even more specifically, the Sydney to Hobart Race. I watched the competitors and interviewed successful ocean racers. And I've sailed with the team of the AFR Midnight Rambler. This immersion has extended over a period of eight years since I first contacted the skipper of the Rambler, Ed Psaltis.

That background gave me a context for understanding the race and a perspective about the ingredients of effective teamwork in ocean racing. But as I began to draw lessons from the race and to write about teamwork, something seemed to be missing. What was missing, I concluded, was that I had never done the Hobart. What I was missing was experience—up close and personal—with the Everest of ocean racing. With some trepidation, I decided to find a way to enter the Sydney to Hobart Race.