Analytical Urban Legends

This assertion is contrary to some analytical urban legends, so let us dispel those now. Years ago, we began hearing extravagant tales of software that would eliminate the need for human analysts. The most popular story involved a data mining episode involving diapers and beer. The gist of the story was that some grocery retailer had turned loose its powerful data mining software on a sales database, and it had come up with an interesting finding. Male customers who came in to buy beer for the weekend also tended to remember that their wives had asked them to buy diapers (some versions of the story switched around the primary shopping intent), so they put both products in their shopping carts. The retailer quickly moved diapers over by the beer (or vice versa), and sales exploded.

We chased this one down, and the most credible version of the story happened at Osco, a drugstore chain. Some of the company’s data analysts do dimly remember seeing a correlation between diaper and beer sales in their stores. But the analysts had told the software where and how to look for the relationship; it wasn’t just stumbled on by an enterprising young computer. Most importantly, the finding was deemed an anomaly, and diapers and beer were never put in proximity in the Osco stores (not all of which could even sell beer).

The legend is worth discussing, however, for a few lessons it provides. While data mining software is a wonderful thing, a smart human still needs to interpret the patterns that are identified, decide which patterns merit validation or subsequent confirmation, and translate new insights into recommendations for action. Other smart humans need to actually take action. When we studied more than thirty firms with a strong analytical capability in 2000, we found that a heavy dose of human skills was present at each of the firms, and the analytical competitors we’ve studied over the years certainly have lots of smart analysts on board.¹

The other key lesson of the diapers and beer legend is that analytics aren’t enough, even when orchestrated by a human analyst. In order for analytics to be of any use, a decision maker has to make a decision and take action—that is, actually move the diapers and beer together. Since decision makers may not have the time or ability to perform analyses themselves, such interpersonal attributes as trust and credibility rear their ugly heads. If the decision maker doesn’t trust the analyst or simply doesn’t pay attention to the results of the analysis, nothing will happen and the statistics might as well never have been computed.

We found another good example of this problem in our previous study of analytical capability. We talked to analysts at a large New York bank who were studying branch profitability. The analysts went through a painstaking study in the New York area—identifying and collecting activity-based costs, allocating overheads, and even projecting current cost and revenue trends for each branch into the near future. They came up with a neat, clear, ordered list of all branches and their current and future profitability, with an even neater red line drawn to separate the branches that should be left open from those that should be closed.

What happened? Nary a branch was shut down. The retail banking executive who had asked for the list was mostly just curious about the profitability issue, and he hardly knew the analysts. He knew that there were many political considerations involved in, say, closing the branch in Brooklyn near where the borough president had grown up, even if it was well below the red line. Analytically based actions usually require a close, trusting relationship between analyst and decision maker, and that was missing at the bank. Because of the missing relationship, the analysts didn’t ask the right questions, and the executive didn’t frame the question for them correctly.

There are really three groups, then, whose analytical skills and orientations are at issue within organizations. One is the senior management team—and particularly the CEO—which sets the tone for the organization’s analytical culture and makes the most important decisions. Then there are the professional analysts, who gather and analyze the data, interpret the results, and report them to decision makers. The third group is a diverse collection we will refer to as analytical amateurs, a very large group of “everybody else,” whose use of the outputs of analytical processes is critical to their job performance. These could range from frontline manufacturing workers, who have to make multiple small decisions on quality and speed, to middle managers, who also have to make middle-sized decisions with respect to their functions and units. Middle managers in the business areas designated as distinctive capabilities by their organizations are particularly important, because they oversee the application of analytics to these strategic processes. IT employees who put in place the software and hardware for analytics also need some familiarity with the topic. We’ll describe each of these groups in this chapter.

Before we go any further, however, it is important to point out that the role of humans in analytics is changing somewhat, and is likely to change more in the near future. One key development is that machine learning and other intelligent technologies are changing how analytical models are generated. A human analyst might be able to generate a few new models per week, but a machine learning system could easily generate tens of thousands of models per week. Thus far, however, machine learning models still need humans to kick them off, point them in the direction of the right data and the variables to be predicted, and ensure that the resulting models make sense. We don’t think that machine learning models have inhibited employment for quantitative analysts and data scientists yet, but they may in the future.

The other technological factor that’s driving change for humans in the world of analytics is automation of decisions and actions. While an autonomous system is unlikely to close a set of bank branches without human intervention, complex decisions and digital tasks can both be performed by machine. They can do things like approve insurance policies, authorize loans, reboot computer servers, and replace and mail a lost ATM card. It’s likely that the decisions taken over by analytics and computers will be tactical and repetitive ones, but these probably constitute the bulk of decisions in many organizations. We don’t think high-level managers will lose their jobs to autonomous systems, but it’s likely that some employees will. The good news here is that an automated decision system is unlikely to ignore an analytical result. Humans who ignore analytics in the future will do so at their own peril.

Senior Executives and Analytical Competition

As if CEOs, presidents, COOs and other senior executives weren’t busy enough already, it is their responsibility to build the analytical orientation and capabilities of their organizations. If the CEO or a significant fraction of the senior executive team doesn’t understand or appreciate at least the outputs of quantitative analysis or the process of fact-based decision making, analysts are going to be relegated to the back office, and competition will be based on guesswork and gut feel, not analytics. Fact-based decision making doesn’t always involve analytics—sometimes “facts” may be very simple pieces of evidence, such as a single piece of data or a customer survey result—but the desire to make decisions on the basis of what’s really happening in the world is an important cultural attribute of analytical competitors.²

For an example, take Phil Knight, the founder and chairman emeritus of Nike. Knight has always been known as an inspirational but intuitive leader who closely guarded the mythical Nike brand. Perhaps needless to say, it didn’t take a lot of analytics to come up with the famous “swoosh.” At the beginning of 2005, however, Knight brought in William Perez, formerly head of S. C. Johnson & Son, as CEO of Nike. Perez, accustomed to the data-intensive world of Johnson’s Wax, Windex, and Ziploc bags, attempted to bring a more analytical style of leadership into Nike. He notes about himself, “I am a data man—I like to know what the facts are. If you come from the world of packaged goods, where data is always valuable, Nike is very different. Judgment is very important. Feel is very important. You can’t replace that with facts. But you can use data to guide you.”³

Perez attempted, for example, to move Nike into middle-tier retail environments, where his data suggested that footwear growth was fastest. In response to arguments from Knight and other Nike executives that such a move would weaken the brand, Perez pointed to companies such as Apple that sell successfully in Walmart without diluting their brands. But these and other clashes eventually led Knight and the board of directors to remove Perez little more than a year later.

The good news is that Perez’s departure only delayed the rise of analytics at Nike. Over the last several years, the company has become much more analytical. It uses data and analytics to influence shoe design, marketing programs, outlet store locations, logistics, and many other types of decisions. Nike has perhaps the world’s largest analytics group that’s focused on sustainability. The company may have gotten there faster if Perez had stayed, but it is definitely moving toward analytical competitor status.

The general lesson here, however, is that if a CEO can’t move the culture in a more analytical direction, a middle or junior manager would have even less of a chance of doing so. In fact, we’ve seen several companies in which fairly senior functional managers—corporate heads of marketing or technology, for example—were trying to bring a more analytical orientation to their firms and faced substantial obstacles. One, for example, a senior vice president of sales and marketing at a technology company, was known as a true “data hound,” bringing piles of statistical reports to meetings and having perfect command of his own (and other managers’) statistics. The sales and marketing organizations slowly began to become more data focused, but for years the overall company culture continued to emphasize chutzpah and confidence more than correlations. Again, that company eventually embraced analytics (it helped that the “data hound” kept getting promoted, eventually becoming president), but a more receptive culture would have allowed it to happen faster.

While there’s no doubt that almost any employee can move an organization in a more analytical direction, it takes top management commitment for a company to become an analytical competitor. In fact, we didn’t find a single stage 4 or 5 analytical competitor where either the CEO or a majority of the senior management team didn’t believe strongly in analytics as a primary competitive resource. Senior executive support is even important at stage 3, when organizations begin to aspire to analytical competition.

Analytical Professionals and Data Scientists

There is an old joke about analytical professionals. It goes this way:

That joke is now totally obsolete, as demand for analytical talent has skyrocketed. Data science, math, and other analytical professionals are being avidly recruited to play key roles in helping companies compete on analytics.

In addition to committed executives, most of the analytical competitors we studied had a group of smart and hardworking analytical professionals within their ranks. It is the job of these professionals to design and carry out experiments and tests, to define and refine analytical algorithms, and to perform data mining and statistical analyses on key data. Analytical pros create the predictive and prescriptive analytics applications used in the organization. In most cases, such individuals would have advanced degrees—often PhDs—in such analytical fields as statistics, data science, econometrics, mathematics, operations research, logistics, physics, and marketing research. As they become more widely available, they are being joined by a new generation of analysts with master’s degrees in applied analytics, informatics, and data science. In some cases, where the company’s distinctive capabilities involve a specialized field (such as geology for an oil exploration firm), the advanced degree will be in that specialty.

One great example of this type of person we found in our research is Katrina Lane. We first encountered her as vice president of channel marketing at Caesars Entertainment. There, Lane had the job of figuring out which marketing initiatives to move through which channels, including direct mail, email, call centers, and so forth. This is a complex field of business that hasn’t been taught in most business schools, so Lane had to figure a lot out on her own. Fortunately, her skills were up to the task. To start with, she has a PhD in experimental physics from Cornell. She was head of marketing for a business unit of the May Department Stores Company and a consultant with McKinsey & Company’s marketing and sales practice. How common is such a combination of skills and experience? Not very, which is why assembling a capable group of analytical professionals is never easy. The rarity of her skills also explains why Lane was promoted to chief technology officer at Caesars, and then recruited to be an executive vice president and general manager of marketing and operations for Consumer Cards at American Express. Now she is the VP of Global Delivery Experience for Amazon. She clearly excels in highly analytical management roles in highly analytical companies.

With her PhD in experimental physics, Lane would today be hired as a “data scientist,” a job that can bring structure to unstructured data, create sophisticated models to analyze it, and interpret the results for their implications for key decisions and business directions. Data scientists, whom Tom and his coauthor D. J. Patil (until recently, the chief data scientist in the White House) described as holding “the sexiest job of the 21st century,” are in hot demand.⁸ Some starting salaries for data scientists exceed $200,000. Their earliest employers were Silicon Valley startups, but now they’re being hired by large traditional firms as well. Procter & Gamble, for example, went from one data scientist in 2013 to over thirty in 2017. GE hired a couple hundred of them for its GE Digital operation in the San Francisco Bay area. There simply aren’t enough to go around.

Even Google, which is one of the most desired employers on the planet right now, has challenges getting this sort of talent. It offers generous salaries, stock options, and what is reputed to be the best cafeteria food anywhere. Yet UC Berkeley professor Hal Varian, who has worked with Google as a consultant since 2002, notes the difficulty of hiring analytical professionals and data scientists there: “One point that I think needs more emphasis is the difficulty of hiring in this area. Given the emphasis on data, data warehousing, data mining and the like you would think that this would be a popular career area for statisticians. Not so! The bright ones all want to go into biotech, alas. So it is quite hard to pull the talent together, even for Google.”⁹

Assuming you can find them, how many of these people are necessary? Of course, the answer depends on what an organization is attempting to do with analytics. In the companies we studied, the numbers range from about a dozen analytical professionals and data scientists to several hundred. GE had a goal of hiring four hundred data scientists for its software and analytics business based in the San Francisco area. We’re not sure of the exact amount the company hired, but there were at least two hundred in this central organization, and now other business units are hiring their own. Procter & Gamble has two hundred or so. Google has about five hundred people with the “quantitative analyst” title, and thousands who do some sort of analytical work.

How are they organized? Most companies have centralized them to some degree, although organizational structures fluctuate over time. Procter & Gamble, for example, took analytical groups that had been dispersed around the organization, and combined them to form a new global analytics group as part of the IT organization. Then it decentralized them a bit while retaining a dotted line to the chief data officer. AIG created a centralized sixty-person science office for advanced analytics, but then decentralized it for greater responsiveness to the business.

Another logical alternative as an organizational home for these high-powered analysts would be the business function that is the primary competitive thrust for the organization. For example, Caesars keeps most of its “rocket scientists” (including Katrina Lane at the time) in the marketing department, because customer loyalty programs and improved customer service are the primary orientation of its analytics.

One argument in favor of central coordination is that at the most advanced stages of analytics, extensive knowledge of specialized statistical methods is required. As a result of what we learned about the companies we studied, we believe it is impractical for these advanced skills to be broadly distributed throughout the organization. Most organizations will need to have groups that can perform more sophisticated analyses and set up detailed experiments, and we found them in most of the companies we interviewed. It’s unlikely, for example, that a “single echelon, uncapacitated, nonstationary inventory management algorithm,” employed by one analytical competitor we studied in the supply chain area, would be developed by an amateur analyst. You don’t learn about that sort of thing in a typical MBA program.

Most analytical competitors we have studied are evolving toward a hub-and-spoke organization reporting to the CDAO. Analysts are allocated among business units, corporate, and functions so they can specialize and work closely with decision makers. The centralized hub is responsible for knowledge sharing, spreading best practices, analytics training, career path development, common standards and tools. Often, the most highly skilled analysts and data scientists are centralized in the hub so they can be strategically deployed to work on the enterprise’s most pressing projects.

Regardless of where the professional analysts are located in their firms, many of the analytical competitors we interviewed stressed the importance of a close and trusting relationship between these analysts and the decision makers. As the head of one group put it, “We’re not selling analytics, we’re selling trust.” The need is for analytical experts who also understand the business in general and the particular business need of a specific decision maker. One company referred to such individuals as “front room statisticians,” distinguishing them from “backroom statisticians” who have analytical skills but who are not terribly business oriented and may also not have a high degree of interpersonal skills.

In order to facilitate this relationship, a consumer products firm with an IT-based analytical group hires what it calls “PhDs with personality”—individuals with heavy quantitative skills but also the ability to speak the language of the business and market their work to internal (and, in some cases, external) customers. One typical set of job requirements (listed on Monster.com for a data scientist in Amazon’s Advertising Platform group) reads:

• PhD in CS [computer science] machine learning, operational research, statistics or in a highly quantitative field
• 8+ years of hands-on experience in predictive modeling and analysis
• Strong grasp of machine learning, data mining and data analytics techniques
• Strong problem solving ability
• Comfortable using Java or C++/C. Experience in using Perl or Python (or similar scripting language) for data processing and analytics
• Experience in using R, Weka, SAS, Matlab, or any other statistical software
• Communication and data presentation skill

Some of these skills overlap with those of traditional quantitative analysts, but some don’t. Data scientists tend to have more computer science–oriented backgrounds, whereas analysts tend to have a more statistical focus. Data scientists are also more likely to be familiar with open-source software and machine learning, and perhaps more likely to have a PhD in a scientific discipline. There are also differences in culture and attitude to some degree. A table of typical differences between these two groups is provided in table 7-1, which is based on a survey Jeanne led in 2014. Over time (and with the popularity of the data scientist title), however, these categories have become somewhat intermingled, and the differences may be diminishing.

TABLE 7-1

Analysts and data scientists: not quite a different species

Whatever the role is called, business relationships are a critical component of it. At Wells Fargo, a manager of a customer analytics group described the relationships his group tries to maintain: “We are trying to build our people as part of the business team; we want them sitting at the business table, participating in a discussion of what the key issues are, determining what information the business people need to have, and recommending actions to the business partners. We want this [analytical group] to be more than a general utility, but rather an active and critical part of the business unit’s success.”¹⁰

Other executives who manage or have managed analytics groups described some of the critical success factors for them in our interviews:

• Building a sustainable pipeline. Analytical groups need a pipeline of projects, client relationships, and analytical technologies. The key is not just to have a successful project or two but to create sustainability for the organization over time. It’s no good to invest in all these capabilities and have the analytical initiative be in existence for only a few years. However, it takes time to build success stories and to have analytics become part of the mythology of the organization. The stories instill a mindset in the company so that decision makers have the confidence to act.
• Relationship to IT. Even if an analytical group isn’t officially part of an IT organization, it needs to maintain a close relationship with it. Analytical competitors often need to “push the frontier” in terms of IT. One analytical group’s manager in a consumer products firm said that his group had been early users of supercomputers and multiuser servers and had hosted the first website for a product (which got thousands of hits on the first day). Exploration of new information technologies wasn’t an official mission for the group but one that the company appreciated. It also helped capture the attention and imagination of the company’s senior management team.
• Governance and funding. How the group is directed and funded is very critical, according to the executives we interviewed. The key issue is to direct analytical groups at the most important problems of the business. This can be done either by mandate or advice from some form of steering committee, or through the funding process. When steering committees are made up of lower-level executives, the tendency seems to be to suboptimize the use of analytical resources. More senior and strategic members create more strategic priorities for analytical professionals. Funding can also lead to strategic or tactical targets. One group we interviewed was entirely paid for by corporate overhead, which meant that it didn’t have to go “tin cupping” for funding and work on unimportant problems that somehow had a budget. Another group did have to seek funding for each project, and said they had to occasionally go to senior management to get permission to turn down lucrative but less important work.

• Managing politics. There are often tricky political issues involved in analytical professionals’ work, ranging from whose projects the group works on to what the function is called. One company called its analytical group Decision Analysis, only to find that executives objected because they felt it was their job to make decisions. The group then changed its name to Options Analysis. It can also be politically difficult simply to employ analysts. As one head of an analytical group put it: “So I go to Market Research and say, ‘I have a better way to evaluate advertising expenditures.’ They don’t necessarily react with joy. It’s very threatening to them. It makes it particularly difficult if you are asking them to pay you to make them look bad!”¹¹

  Heads of analytical groups have to be sensitive to political issues and try to avoid political minefields. The problem is that people who are good at analytics do not often have patience for corporate politics! Before any analysis, they should establish with the internal client that both client and analyst have no stake in any particular outcome, and they will let the analytical chips fall where they may. CEOs can help their analytical professionals by making it clear that the culture rewards those who make evidence-based decisions, even when they go against previously established policies.

• Don’t get ahead of users. It’s important for analytical professionals to keep in mind that their algorithms and processes must often be implemented by information workers, who may be analytically astute but are not expert statisticians. If the analytics and the resulting conclusions are too complex or laced with arcane statistical terminology, they will likely be ignored. One approach is to keep the analytics as simple as possible or to embed them into systems that hide their complexity. Another is to train users of analytical approaches as much as possible. Schneider National’s analytics group has offered courses such as “Introduction to Data Analysis” and “Statistical Process Control” for users in various functions at the company. It’s not a formal responsibility for the group, but the courses are popular, and the group feels it makes their job easier in the long run.

Analytical Amateurs

Much of the daily work of an analytically focused strategy has to be implemented by people without PhDs in statistics or operations research. A key issue, then, is how much analytical sophistication frontline workers need in order to do their jobs. Of course, the nature and extent of needed skills will vary by the company and industry situations. Some firms, such as Capital One, hire a large number of amateur analysts—people with some analytical background (perhaps MBAs), but mostly not PhD types. At one point, when we looked at open jobs on Capital One’s website, there were three times as many analyst openings as there were jobs in operations—hardly the usual ratio for a bank. According to its particular analytical orientation, a company simply needs to determine how many analytical amateurs it needs in what positions. Some may verge on being professionals (we call them analytical semi-professionals); others may have very limited analytical skills but still have to work in business processes that are heavily based on analytics.

The Boston Red Sox’s situation in 2003, which we described in chapter 1, is an example of needing to spread analytical orientations throughout the organization. For more business-oriented examples, we’ll describe two organizations that are attempting to compete on supply chain analytics. One, a beer manufacturer, put in new supply chain optimization software to ensure that it manufactured and shipped the right amount of beer at the right time. It even created a new position, “beer flow coordinator” (if only we had such a title on our business cards!) to use the system and oversee the optimization algorithms and process. Yet the company’s managers admitted that the beer flow coordinators didn’t have the skills to make the process work. No new people were hired, and no substantial training was done. The new system, at least in its early days, was not being used. The company was expecting, one might say, champagne skills on a beer-skills budget.

At a polymer chemicals company, many of the company’s products had become commoditized. Executives believed that it was important to optimize the global supply chain to squeeze maximum value and cost out of it. The complexity of the unit’s supply chain had significantly increased over the previous couple of years. Responding to the increased complexity, the organization created a global supply chain organization, members of which were responsible for the movement of products and supplies around the world. In the new organization, someone was responsible for the global supply chain, there were planning groups in the regions, and then planners in the different sites. The greatest challenge in the supply chain, however, involved the people who did the work. The new roles were more complex and required a higher degree of analytical sophistication. The company knew that the people performing the previous supply chain roles didn’t have the skills to perform the new analytical jobs, but it kept them anyway. At some point, the company plans to develop an inventory of skills needed and an approach to developing or hiring for them, but thus far the lack of skills remains a bottleneck in the implementation of its new logistical process.

When a company is an analytical competitor, it will need to ensure that a wide variety of employees have some exposure to analytics. Managers and business analysts are increasingly being called on to conduct data-driven experiments, interpret data, and create innovative data-based products and services. Many companies have concluded that their employees require additional skills to thrive in a more analytical environment. An Avanade survey found that more than 63 percent of respondents said their employees need to develop new skills to translate big data analytics into insights and business value.¹² Anders Reinhardt, formerly head of Global Business Intelligence for the VELUX Group—an international manufacturer of skylights, solar panels, and other roof products based in Denmark—is convinced that “the standard way of training, where we simply explain to business users how to access data and reports, is not enough anymore. Big data is much more demanding on the user.”¹³

To succeed at an analytical competitor, information workers and decision makers need to become adept at three core skills:¹⁴

• Experimental: Managers and business analysts must be able to apply the principles of scientific experimentation to their business. They must know how to construct intelligent hypotheses. They also need to understand the principles of experimental testing and design, including population selection and sampling, in order to evaluate the validity of data analyses. As randomized testing and experimentation become more commonplace in the financial services, retail, and telecommunications industries, a background in scientific experimental design will be particularly valued. Google’s recruiters know that experimentation and testing are integral parts of their culture and business processes. So job applicants are asked questions such as “How many tennis balls would fit in a school bus?” or “How many sewer covers are there in Brooklyn?” The point isn’t to find the right answer but to test the applicant’s skills in experimental design, logic, and quantitative analysis.
• Numerate: Analytical leaders tell us that an increasingly critical skill for their workforce is to become more adept in the interpretation and use of numeric data. VELUX’s Reinhardt explains that “Business users don’t need to be statisticians, but they need to understand the proper usage of statistical methods. We want our business users to understand how to interpret data, metrics, and the results of statistical models.” Some companies, out of necessity, make sure that their employees are already highly adept at mathematical reasoning when they are hired. Capital One’s hiring practices are geared toward hiring highly analytical and numerate employees into every aspect of the business. Prospective employees, including senior executives, go through a rigorous interview process, including tests of their mathematical reasoning, logic, and problem-solving abilities.
• Data literate: Managers increasingly need to be adept at finding, manipulating, managing, and interpreting data, including not just numbers but also text and images. Data literacy is rapidly becoming an integral aspect of every business function and activity. Procter & Gamble’s former chairman and CEO Bob McDonald is convinced that “data modeling, simulation, and other digital tools are reshaping how we innovate.” And that changed the skills needed by his employees. To meet this challenge, P&G created “a baseline digital-skills inventory that’s tailored to every level of advancement in the organization.” The current CEO, David Taylor, also supports and has continued this policy. At VELUX, data literacy training for business users is a priority. Managers need to understand what data is available, and to use data visualization techniques to process and interpret it. “Perhaps most importantly, we need to help them to imagine how new types of data can lead to new insights,” notes Reinhardt.¹⁵

Depending on the business function, additional expertise may be needed. Most IT people, for example, should have some sense of what analyses are being performed on data, so that they can ensure that IT applications and databases create and manage data in the right formats for analysis. HR people need to understand something about analytics so that they can hire people with the right kinds of analytical skills. Even the corporate legal staff may need to understand the implications of a firm’s approach to analytical and automated decision making in case something goes awry in the process.

Firms that have upgraded the analytical skills of employees and managers are starting to see benefits. For example, at a consumer products firm with an analytical strategy, they’re seeing a sea change in middle managers. Upper middle management has analytical expertise, either from mathematical backgrounds or from company experience. Two of the central analytical group’s key clients have new managers who are more analytical. They were sought out for their analytical orientations and have been very supportive of analytical competition. The analytical managers are more challenging and drive the professional analyst group to higher levels of performance. The senior management team now has analytical discussions, not political ones.