CHAPTER 5
Developing Intuition forNumbers

It is better to be roughly right than precisely wrong.

John Maynard Keynes

An important question we're often asked about the Quantitative Intuition (QI)™ approach is whether you can actually learn intuition. It is clear that you can teach the Q in QI, but isn't intuition something you either have or don't have? Can you really learn to be intuitive about numbers? Quite simply, the answer is “yes,” you can develop intuition, and in this chapter, we show you how.

We first discuss how to resist the urge for certainty and the benefit in rough approximations. We then demonstrate how business leaders can learn numbers intuition by practicing the power of approximation. We show how to look at a number and, relatively quickly—by using available related figures or back‐of‐the‐envelope calculations—assess whether that number is likely to be within the ballpark of the true figure. Approximations allow decision‐makers to get a practical check on the numbers presented to them and better understand the drivers behind the figures. The habit of approximating translates into intuition. Rather than relying on complicated calculations that promise exact accuracy, you develop a feel for what seems about right. You also build the trust and confidence that underlie intuition.

In our work, we often observe that decision‐makers are swayed by statisticians and analysts to require universal, predefined high levels of accuracy. Of course, some decisions do indeed require a high level of accuracy, but many do not. The QI framework tells us that decisions (not data) should guide the process. For every decision, we should ask ourselves “What accuracy level is needed to make this particular decision?” For many business decisions, particularly when evaluating whether to pursue a project or idea at an early stage, rough estimates are entirely sufficient.

The Power of Approximations

Oded had the occasion recently to observe a group of MBA graduate students presenting their entrepreneurial idea of opening a farm‐stay in Guatemala to a panel of experts. Farm‐stays are vacation resorts where guests stay on a farm, enjoy the countryside, and take part in farm life. This sort of vacation has become increasingly popular.

The students' slide deck was packed with data. To establish the feasibility of their idea, the team adopted a top‐down approach, starting with a measure of the overall size of the world tourism market, then narrowing down that figure to the tourism market in Central America, and subsequently inferring what the size of the market in Guatemala might be. Using that figure as a basis, they then estimated the proportion of tourists to Guatemala who would prefer a farm‐stay experience over something else. They then used this to calculate the likely demand for services offered by their particular business. Multiplying by the average likely cost of a stay in the farm for one family, they arrived at an expected revenue of $32 million a year for their farm‐stay business in Guatemala.

As they went through their pitch, the team was grilled by the panel members on almost every aspect of the analysis. Is Guatemala similar to Costa Rica in terms of counting tourists to Central America? Aren't the segments that visit Costa Rica and Guatemala quite different? Where did you get the numbers for the share of preference for farm‐stays? Does your analysis include eco‐tourism? were only a few of the numerous questions fired at the team. Seeking certainty and accuracy, the panel members left almost no number unchallenged. It was a grueling 25 minutes for the student team.

As Oded tells it, sitting at the back of the room, and having practically no experience of or expertise in the tourism industry, particularly in Central America:

I had little intuition about whether the team's calculation and the eventual projected revenue was high or low.

Rather than strive for certainty by scrutinizing every number, as my colleagues on the panel seemed to be doing, I decided to take a different approach. I started at the end, with the projected estimate of $32 million a year in revenue. The question I asked myself was not whether that figure was accurate. In fact, considering that it was a projected number for a project that wouldn't launch for at least a couple of years, I knew that it was not accurate. Instead, I asked myself whether this number was likely to even be in the ballpark of what a farm‐stay in Guatemala would be able to make in a year. If the team's estimated revenue was indeed reasonable, this could be a great business. But was the figure way too small, way too large, or about right?

By reformulating the question—leaving aside the accuracy of the calculations and focusing instead on the single most important number in the presentation, the projected revenue—I felt more comfortable relying on my intuition. I asked myself what my gut told me about this number, and then I turned to an approximation.

Oded avoided analysis paralysis. He didn't concern himself with every aspect of the group's thinking and calculations. Focusing only on the essential question, Oded's IWIK™ (see Chapter 2) was: “I wish I knew whether the team's revenue estimate was even within a reasonable range.”

Even as a total tourism industry outsider, Oded knew a few figures—even erring on the high side—a night at a rustic, rural farm‐stay in a country like Guatemala should never cost more than $300 per room, and he knew that such a property was unlikely to have more than 50 rooms.

Assuming that each room is occupied for 250 days of the year and doing a quick back‐of‐the‐envelope calculation in my head, I came up with a yearly revenue of just $3.75 million. The team's calculation of $32 million wasn't even close. Even if I had been more generous on price per night, number of rooms, or occupancy rate, I still wouldn't have come close to the team's reported number.

Using fifth‐grade multiplication allowed Oded to circumvent his lack of experience in the hospitality industry and develop an intuition for the numbers presented.

By resisting the temptation to strive for certainty, and by challenging yourself to establish whether the numbers looked in any way realistic, you can save yourself time and energy, and focus on the issues the matter.

Learning to Approximate

As decision‐makers, we often find ourselves flooded with figures that have varying degrees of credibility, so it is crucial to be able to make first‐order approximations. Making such approximations does not require you to be a math whiz; all it takes is having a few numbers at your fingertips and basic multiplication and division skills. The idea behind such approximations is to provide a practical check on a number you are faced with. While approximations are almost certainly incorrect, they provide an easy and simple calculation to check for possible errors and uncover unreasonable assumptions that might have led to a figure that is either far too large or far too small.

How can decision‐makers acquire intuition for numbers? Experience helps, but to quote the famous German poet Heinrich Heine, “Experience is a good school. But the fees are high.” Through our decades of experience, we've learned that business leaders can often make up for a lack of experience, and complement the experience they do have, by grasping a few key numbers about their business and learning to approximate. Sometimes, all you really need is a guesstimate. The writer A.A. Bell defines guesstimate as: “better than a guess but not as guaranteed as an estimate.” To practice the art of guesstimation, we need an approach that can make the most of a little data.

Scientists often use first‐order approximations to get a rough estimate of an answer before turning to more sophisticated analyses to obtain a precise solution. The Italian Scientist Enrico Fermi devised an approach to make such rough approximation (see the call‐out box, “Enrico Fermi”). Fermi believed that the starting point to solving any complex problem is using shortcut simplifications to make meaningful approximations. The Fermi approach allows us to estimate quantities that seem difficult or even impossible to estimate at first, like how many piano tuners there are in Chicago (see call‐out box “Guesstimating the Number of Piano Tuners in Chicago”). His approach is to break down a problem into smaller problems, or factors, that are easier to estimate, and then to piece these together to get a rough estimate of the number you are trying to approximate. If scientists, who are much more comfortable than most of us in applying sophisticated methods, often get a rough estimate before turning to more sophisticated analyses, we should feel comfortable doing so as well.

Approximating in Practice

Approximation is also an excellent tool when we simply don't have all the numbers we'd need for an accurate calculation. For example, imagine that you are in the business of developing disposable towels to be used in delivery rooms for wrapping babies right after delivery to maintain their body temperature. You'd probably want to estimate the number of babies delivered in the United States per day. Estimating this number will allow you to approximate your expected revenue if you were able to sell to every hospital, or to a certain share of hospitals. How would you go about estimating this number? Some people might start by simply attempting to guess the number itself, but unless you have seen a similar figure before, you probably won't have a good enough gut feeling to make a guess.

Following Fermi's estimation method, you'd want to break the problem into smaller factors that are easier to approach. There are multiple ways you could go about breaking down this quantity. You can estimate the number of maternity wards in the United States and the number of deliveries per ward, but the number of maternity wards is probably as difficult to estimate as the number of babies born per day in the United States. So, let's start with possibly relevant figures that we do know. We definitely know how many days there are in a year (365). We probably also have a good idea about the number of people living in the United States (approximately 330 million), and let's say that approximately half of them are women. We probably also want to guesstimate how many children an average family has. Looking around at your friends, we can probably guess about two. We have almost all of the ingredients to try and address the problem. You can estimate the number of women of childbearing age and assume that within the 15–20 years of childbearing age, women deliver on average two babies. That would probably get you pretty close to a good estimate.

Here are a couple of simpler routes to breaking the problem into factors. If, on average, every woman in the United States delivers two babies during her 80 years of average lifetime, we can estimate how many babies are delivered each day. For that, we need to know the number of women in the United States, the average lifetime of a woman in the United States, and how many babies an average woman delivers during her lifetime. From there it's simple multiplication and division.

upper Number of women in the normal upper U period normal upper S period equals 165 normal upper M
upper Babies delivered per women equals 2

upper Average life time of normal a women in days equals 80 years asterisk 365 d a y s
upper Number of deliveries per days equals 165 normal upper M asterisk 2 slash left-parenthesis 80 asterisk 365 right-parenthesis approximately-equals 11 comma 300

An alternative approach would be to say that the 330 million people living in the United States with an average age of 78 years had to be born on a particular day of the year. Leading to this calculation:

upper Number of people in the normal upper U period normal upper S period equals 330 normal upper M
upper Number of days the y live equals 365 asterisk 78 days
upper Number of deliveries per days equals 330 normal upper M slash left-parenthesis 365 asterisk 78 right-parenthesis approximately-equals 11 comma 6 00

The actual number in 2020 was 9,877 births per day, so both estimates are pretty close. In fact, they are both about 15% larger than 9,877. That is totally fine. Keep in mind why we estimated this number. We are not looking for the exact figure, just for estimates that are in the same ballpark. We can later conduct a study or contact the department of health for the exact number. By the way, by the time you finish reading this sentence, another three babies will be born in the United States, so do you really need to know the exact number?

Why Guesstimation Works

In estimating the number of babies delivered, we made a lot of simplifications. We ignored a lot of details. We didn't take into account the uneven spread of the population over years and the baby boomers, the increase in life expectancies over the years, or the fact that because of their higher life expectancy, the share of women living in the United States is higher than 50%. But recall that we are looking for the Goldilocks measure—the too big, too small, or about right estimate—not an accurate estimate.

In breaking down a problem into smaller problems, we ignore the details and focus only on the major factors. When you first begin using this estimation method, it may feel odd to ignore so many details, but as you keep practicing and see close enough estimates, a weight will lift off your chest, and you will become more comfortable with the process, and with inaccuracy. In our striving for certainty, we forget the cost‐benefit of obtaining the so‐called accurate estimates. If you do not need an accurate estimate at this point to make a decision, getting a guesstimate is the most effective and efficient course of action.

Why does this method of estimation work? One of the reasons is that the estimates you use for each of the smaller problems in your breakdown are often much more accurate than your best guess for the eventual number. In addition, the overestimates and underestimates you make in each of the smaller problems will tend to cancel each other. For example, in the baby deliveries problem, if you erroneously underestimated the life expectancy to be 70, and you overestimate the number of kids per household to be three, you nearly cancel the two errors. Breaking down the unknown and uncertain figures to figures for which you have greater certainty has another benefit, too: it helps you develop and gain confidence in your numbers intuition.

In a business context, when using Fermi's approach to assess a figure presented to you, you can make adjustments to the factors as indicated by your problem. For example, if you believe that the number presented to you is too large, in making your approximations, you should be generous with your estimates for figures you are unsure about. More generous estimates will enable you to see if it is possible to get close to the presented number. Similarly, if you think the number is too small, be conservative with your estimates. Your goal is to evaluate if that number is possible, not whether it is correct. If you are conducting the approximation for your own business, for example, to estimate the potential market, be honest with your estimates. If you start choosing generous estimates for each step of the calculation, the errors will compound, not cancel each other, and you will inflate your final estimate, possibly substantially.

If in one of the steps of the estimation you are faced with the need to estimate a number about which you still have little knowledge, use upper and lower bounds for comparatives. For example, if you need to estimate the population of Chicago and you don't know what it is, but you happen to know the population of San Francisco and New York, and you know that Chicago's population lies between the two, take an average of the populations of San Francisco and New York; it will probably get you pretty close.

We have run numerous workshops employing the guesstimation approach to estimate various quantities about which participants had little knowledge (e.g., number of hot dogs consumed at U.S. Major League Baseball games per year, gallons of water flowing through the Niagara Falls in a second). Time after time we are amazed by how close participants are in approximating the numbers using this approach.

Getting Comfortable with Approximations

If the idea of using a guesstimate in decision‐making is making you uncomfortable, we're not surprised. As decision‐makers and business leaders, we often frown on rough estimates and approximations. We are wary of inaccuracies. We have been taught from a young age, particularly when it comes to math and numbers, to strive for accuracy. When the projections for next year's revenue figures are presented to us with three digits after the decimal point, we get a feeling of comfort—even though we know we can only realistically hope to project these figures in orders of thousands or tens of thousands at best.

Consider Approximation in Context

QI informs us that the nature of the decision should determine our approach, not the availability of data. As we see in the farm‐stay example, not all problems require accurate figures. Often an approximation is enough.

The type of decision you're making will determine the degree of accuracy you need. When the Food and Drug Administration (FDA) decided to approve the Pfizer and Moderna vaccines for Covid‐19, even for emergency use, the numbers from the clinical trials were heavily scrutinized, evaluated, and re‐evaluated. However, when the CEO of a startup that's at an early stage of funding is asked by a venture capitalist how many people they expect the company to employ in three years' time, a super accurate response, like 237, wouldn't only sound odd, it would be unnecessarily specific. A rough ballpark range of 200 to 300 would be much more appropriate.

Striving for certainty when only rough estimates are needed is not only inefficient, but it is also risky. The presentation of seemingly precise estimates may give the decision‐maker the erroneous perception that they are basing their decisions on certain figures. Consider a spreadsheet providing a projection of next year's sales. Because the numbers are based on many assumptions and multiplication of multiple factors like projected unit sold, price, and conversion rates, the eventual figure is often presented with a few digits after the decimal. This practice should be considered inappropriate. The presentation of a projected number with a high degree of uncertainty around it as a precise estimate is confusing, and even worse, misleading. Instead, we recommend matching the degree of reporting of figures with the corresponding level of certainty about these figures. Projections for next year's sales that are probably accurate only at the level of tens of thousands of dollars should be rounded to the nearest 10 thousandth.

In our work with business leaders, we often use the T‐shirt sizing analogy to help people feel comfortable with guesstimates. The clothing industry has long recognized the need to match the metrics of garment sizing to the customer's need for accuracy. When it comes to clothes like shoes or dress shirts, the customer seeks a high level of accuracy on the fit of the item, and accordingly, the sizing measures are often quite nuanced moving in small increments. However, when choosing which T‐shirt to buy, relatively few sizes—small, medium, large, and extra‐large—usually suffice.

We should adopt a similar approach to our business decisions. When asked how much time it would require to complete a relatively complex task, an engineer may be reluctant to offer an accurate estimate. It may be too early to tell. And then there are the risks of inaccuracy: Underestimating the time may result in false expectations and the perception of underperformance, and overestimating may lead to the perception of lowballing. Rephrasing the problem as a T‐shirt sizing problem and asking the engineer to categorize the time required as extra small, small, medium, large, or extra‐large, could relieve the engineer's fear of being inaccurate. Looking at the problem from the decision‐maker's perspective, in many cases, when we ask how long a project would take, the decision at hand is the prioritization of tasks. We need to know whether the task will take hours, days, or weeks. Phrasing the question at the appropriate level of accuracy, for example, “Will this task take you hours, days, or weeks to complete?” will help ensure that you get the answers you need to make decisions in an efficient way.

In weighing the appropriateness of approximation, we should consider the context. In the early stages of a decision‐making process, ballpark estimates are particularly useful to help make sense of the problem or evaluate the feasibility of moving forward before committing to bigger time and monetary investments. Approximations can also be used as part of the data interrogation stage, as discussed in Chapter 4, where a ballpark figure can help identify possible problems with the analysis early on, as we see in the farm‐stay example. Approximation can be used in later stages of the decision‐making process to better understand the economics behind an overall business. We encourage business leaders to conduct back‐of‐the‐envelope calculations regularly in order to develop their own intuition for numbers and strengthen their critical reasoning and quantitative intuition in the domain in which they operate.

You may have heard data analysts and colleagues who are proficient in statistics often talk about providing precise estimates with tight confidence intervals of 95% or a p‐value of less than 0.05. This means that 1 out of 20 times that you make the statement implied by the data findings, you will be wrong. But applying the QI framework, and the lens of decision‐driven journeys, we can recognize that there is no uniform required level of confidence that is independent of the decision at hand.

For many business decisions, such as choosing version A of an ad over version B, being wrong 1 out 10 times, or even 1 out of 5 times, rather than 1 out of 20 times, would be great. On the other hand, being wrong 1 out of 20 times is obviously too much for a decision such as whether to launch the space shuttle Challenger on a night in which the temperature in Florida was unusually cold, at only 26 degrees Fahrenheit. Hopefully, the lesson is clear: the confidence level that you require from the data and analyses should not just be a matter of sample size and statistics. It should also account for the context of the decision and the risk involved in making erroneous decisions. Consideration of context is unlikely to come from the analyst or data scientists. These issues are a matter of deep business understanding, and, as you develop it, of intuition.

In addition to the context of the decision, your experience and intuition should also matter in determining the desired level of accuracy. If a specific analysis provides evidence or insights that point in the same direction as many other pieces of evidence that we have seen before, we will be inclined to act on such insights, even if the statistical confidence level provided for that analysis is rather low. On the other hand, a study that shows single evidence for slower melting of a specific glacier—contradicting years of research and statistical evidence demonstrating the increasing temperature of the earth—would require a higher confidence level to be considered convincing.

Statistical Significance Doesn't Mean Managerial Relevance

You should also consider the difference between statistical significance and managerial relevance. A relationship between two variables can be statistically significant but managerially totally irrelevant, or the other way around, profoundly meaningful even if it does not reach statistical significance. Numerous times analysts have come to us proudly celebrating their achievement of finding a statistically significant relationship between two variables. In many of these cases we have, unfortunately, had to cut the celebration short, informing the analysts that despite the result being most likely true from a statistical point of view, its meaning for the business is minimal, either because the relationship itself does not matter or because the absolute magnitude of the effect is too small to make a real difference.

Consider, for example, exploring the relationship between customer service level and customer retention. Your analyst found a statistically significant relationship between the tenure of the call center agent with the firm and customer retention, concluding that we should work harder to keep our experienced call center agents. But looking beyond the statistical significance, we see that an extra year of tenure of the call center agent is associated with an additional two days of customer lifetime. While the result may be statistically significant it may be managerially irrelevant for several reasons. First, are two extra days of customer lifetime meaningful to the company? Second, even if the increase in customer retention is meaningful, with an average tenure of a call center agent in our company of only three months, can we really hope to increase tenure by a full year? Always look beyond the statistical significance into the effect size and managerial meaning of the findings for the business.

Guesstimation in the QI Framework

Too often in business we fall into the rabbit hole of striving for certainty, forgetting to weigh the cost of pursuing accuracy against its possible benefit. If you are at a point in your decision‐making where you do not need an accurate estimate to make a decision, guesstimating is the most effective and efficient course of action.

We advocate using the guesstimate approach in every step of the QI framework. It can be used in the early stage of the process before any data has been collected to get rough estimates before getting into the analysis. It can be used when interrogating the data to pressure test models or estimate profit projections. It can also be used at the level of synthesis to question numbers presented to us. We have also seen the power of this approach to develop numbers intuition, even in situations and domains where we lack experience. Trusting inaccurate estimates may seem at first to go against everything you have learned in school about math, but as you start seeing the value of this approach, you will develop not only intuition but also greater comfort with approximation and uncertainty. The more you practice guesstimation, the closer you will get to the power of QI.

Key Learnings ‐ Chapter 5

  • Begin by determining the level of accuracy needed for the decision at hand. For many decisions, rough approximations are all you need.
  • When evaluating a number presented to you, before jumping into complicated analysis, start by getting a rough estimate for whether the figure is in the ballpark.
  • Use back‐of‐the‐envelope calculations to save time and understand the problem.
  • Follow Fermi's approach to obtain rough estimates:
    • Break down the problem into small subproblems with factors that are easier to estimate.
    • Start with facts that you may know.
    • For figures you don't know, remember the Goldilocks rule: too small, too big, about right.
    • Assess possible ranges or comparables.
    • Don't worry about small details; you're looking for rough estimates.
    • Use worst‐case scenario bounds but be honest with your estimates.
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset