The Inverse Normal Problem

While we now can easily compute probabilities P(x < a) for given values of a, we sometimes want to do just the opposite: given a probability p, find a cut-off value a such that P(x < a) = p. Excel has just the function for us, as usual.

Of course, once we can find values of a such that P(x < a) = p, we can also find cut-off values if the prescribed probability has a different form. Perhaps an example will clarify this:

For the first question, we know right away that a must be less than 0 because the distribution is normal with mean 0 and standard deviation 1. Thus, if a probability of the form P(x < a) wants to be less than 50 percent, a must be negative. In fact, a = NORMINV(0.4, 0, 1) = −0.2533. Indeed, we can check that NORMDIST(−0.2533, 0, 1, true) = 0.4, which is of course the inverse of the problem.

If the variable x was N(4, 1.5) instead of the standard normal and we again wanted to find a such that P(x < a) = 0.4, then it is easy to see that a has to be less than the mean of 4. In fact, a = NORMINV(0.4, 4, 1.5) = 3.6200. We could verify this again using NORMDIST but we will leave that to you.

Finally, with a little imagination and the picture of the normal distribution in our mind we can figure out that to find b such that P(x > b) = 0.05 is equivalent to P(x < b) = 0.95 so that b = NORMINV(0.95, 5, 1) = 6.6448. Indeed, to double-check: P(x > 6.6448) = 1 − P(x < 6.6448) = 1 − NORMDIST(6.6448, 5, 1, true) = 0.05.

Normal Distribution and Its Standard Deviation

While the mean of a normal distribution is easy to see (it is the line of symmetry through the top of the mountain), it seems harder to visualize the standard deviation. It is relatively simple to decide which of the two normal distributions has the smaller variance, but as it turns out even if you have only one normal distribution you can “see” the standard deviation.

This rule can be used to verify whether a given distribution of data is normal or not: check how much of the data is within one, two, and three standard deviations of the mean and compare it with the three-sigma rule of thumb: if there is an approximate match, the distribution is likely normal.

Figure 4.11 Normal distribution and standard deviation

We want to find:

,

which works out to 0.9545. This matches with our observation that the interval (m − 2s, m + 2s) contains approximately 95 percent. Thus, we expect 0.95 ⋅ 500 = 475 bags will have the desired weight.

Incidentally, the preceding rule explains why we can approximate the standard deviation s as range/4, as we saw in Chapter 3: the interval m − 2s to m + 2s contains approximately 95 percent of the data, or in other words, the strip from m − 2s to m + 2s has a width of 4s and contains 95 percent of the data, approximately. Thus, 4s ≈ range or s ≈ range/4.

Converting to z-Scores

It turns out that you can easily convert one normal distribution into another. This is particularly handy when converting an arbitrary normal distribution to the standard normal N(0,1).

This formula allows us to compute probabilities of normally distributed variables in (at least) two ways.

For part (a) we compute as usual:

By the transformation formula the variable is N(0, 1). But if x = 2 then the z-score is and the z-score of x = 6 is . Thus:

Thus, as long as you know how to compute probabilities of the standard normal distribution you can actually compute probabilities of any normal distribution. Therefore, Excel includes a special function to compute probabilities of the standard normal.

The NORMSDIST has the advantage that it is somewhat simpler to use but offers no other benefits. Thus, we will stick with NORMDIST(Z, 0, 1, true) as a reminder that the standard normal distribution has mean 0 and standard deviation 1.

Discrete and Continuous Random Variables

Previously we have used the term “variable” without properly defining it; we relied on common sense. Since we are currently adding a solid foundation to our discussion anyway, we might as well do the same for our most basic terminology.

Let us say we are tossing a single coin once. A random variable needs to assign numbers to the events in the sample space. Thus, we define a random variable x by saying, for example, that x({H }) = 0 and x({T }) = 1. If we toss a coin twice, a random variable could count the number of H’s, so that x({T,T }) = 0, x({T,H }) = x({H,T }) = 1, and x({H,H }) = 2. These two variables are discrete. As an example for a continuous variable, consider an experiment that measures the height of people. A random variable x could simply be the height of a person in inches. For discrete random variables it is convenient to define them via a table of values including their probabilities, while continuous random variables are often represented as the graph of a function called the probability density function.

Mean and Standard Deviation for Discrete Random Variables

Of course we can compute the mean or standard deviation of discrete random variables. It is similar to computing those parameters for frequency tables but we need to take into account that the distinct values of our random variable can occur with different probabilities.

The standard deviation is, as usual, the square root of the variance:

This looks pretty intimidating but once you work through an example everything should clear up and you should have no problems. Note that interpreting the mean of a random variable as the expected value is particularly interesting.

First, we will convert the information into our new lingo: we define the random variable x to measure how much profit a pizzeria makes. Thus, x has four distinct values with the probabilities as shown in column 2 of Table 4.3.

Table 4.3 Finding the expected profit opening a pizzeria

x (Profit) ($)	P(x = xi)	xi P(x = x1)	x2i P(x = x1)
−35,000	0.3	−10,500	367,500,000
0	0.4	0	0
25,000	0.2	5,000	125,000,000
95,000	0.1	9,500	902,500,000
		4,000	1,395,000,000

To find the mean, or the expected value, we multiply xi P(x = x1) and add column 3 to the table. Since we also need to find the standard deviation, we add one more column containing xi2 P(x = x1). Then we find the total of column 3, which will be the expected value of x. Thus, the expected value of x is $4,000, which means that statistically speaking you can expect a profit of $4,000 if you open the pizzeria.

To compute the variance (and hence the standard deviation), we add up the fourth column and use the preceding formula to compute the variance: s2 = 1,395,000,000 − 16,000,000 = 1,379,000,000 so that the standard deviation becomes $37,134.89.

Mean and Standard Deviation for Continuous Random Variables

Defining mean and standard deviation for continuous random variables requires integration of functions—that is, areas under curves—and is generally beyond the scope of this text. Still, for completeness, we list the definitions here as well.

The standard deviation is, as usual, the square root of the variance: .

Even though we do not know how to integrate, here is a relatively simple example.

The random variable x can take any value between 0 and 360: it is therefore a continuous variable. Since you are spinning randomly, every angle is equally likely, so x is called a uniformly distributed random variable. The probability density function for x must be constant, since every value between 0 and 360 is equally likely: p(x) = c for 0 ≤ x ≤ 360 (see Figure 4.13).

Figure 4.12 A continuous “wheel of fortune”

Figure 4.13 Uniform distribution

We know that the total probability has to be 1, as always, so that the area of the rectangle with width 360 and height c must be 1. Therefore, 360 c = 1, so that

. Now we can find the mean and variance:

,

.

Other Distributions

There are many different distributions. In fact, any function p(x) that is non-negative for all x and with the total area under the curve being 1 can generate a probability distribution. We already introduced the normal distributions and worked extensively with them, and in the previous section we introduced a uniform distribution for 0 ≤ x ≤ 360. Now we will introduce two additional ones.

You can see their graphs in Figures 4.14 and 4.15.

The t-distribution looks similar to the standard normal distribution but its peak is not quite as high whereas its tails are wider. If the degree of freedom is high, the t-distribution is nearly identical to the standard normal distribution. The F distribution, on the other hand, looks completely different (see Figure 4.14). In particular, it has no axis of symmetry. We will need these distributions in later chapters, at which point we will also explain the significance of the degree of freedoms. Right now we just want to familiarize ourselves with new ways to compute probabilities.

Note that the Excel definitions of both TDIST and FDIST give the probabilities at the tail end of the distribution whereas NORMDIST gives the probability to the left of x. See Figure 4.15.

Figure 4.14 t-distribution versus standard normal (left) and two F distributions (right)

Figure 4.15 One- and two-tailed t-distribution versus standard normal and F distribution

Assuming x is distributed according to a t-distribution with df = 6, we have . On the other hand, P(|x| ≤ 1) = 1 − TDIST(1,6,2) = 0.8220.

To verify that a t-distribution with high degrees of freedom is about equal to the standard normal distribution, we compare TDIST(1,1000,1) = 0.15878 with and repeat those calculations for different values of x. You will find that the probabilities agree very well indeed.

If x is distributed according to the F distribution with df 1 = 10 and df 2 = 2 then . On the other hand, if x is N(0, 1) then .

The Inverse Probability Problem

We have seen that before that we can also solve the inverse probability problem: instead of finding the probability P(x < a) for a given value of a, we compute that value of a that results in a given probability p = P(x < a). If x is normal, then we can use the Excel function NORMINV. Similarly, Excel offers the functions TINV and FINV that are similar to NORMINV, but with slight differences in the interpretation of their inputs.

• If x is distributed according to a t-distribution with degrees of freedom df, then the Excel function TINV(p, df) returns that value a such that P(x < −a) + P(x > a) = P(|x| > a) = p.
• If x is distributed according to an F distribution with degrees of freedom df 1 and df 2, then the Excel function FINV(p, df 1, df 2) returns that value a such that P(x > a) = p.

The functions TINV and TDIST are inverse of each other, as are FINV and FDIST.

The Central Limit Theorem

In the “Introduction” section we saw that we can use frequency distributions to compute probabilities of various events. Then we determined that we could use various normal distributions as a shortcut to compute those probabilities, which was very convenient. Using that technique we were able to compute all kinds of probabilities just based on the fact that we knew the mean and sample standard deviation of the distribution. We had to assume, however, that the (unknown) distribution of the variable in question was normal with the computed mean and standard deviation as parameters.

As it turns out, there is some mathematical justification for that; it says, in effect, that most distributions—in some sense—are “normal.” That theorem, called the Central Limit Theorem, is one of the corner stones of statistics. It has many practical and theoretical implications, some of which we will explore in subsequent chapters.

In this course we will simply state the theorem without any proof. In more advanced courses we would provide a justification or mathematical proof, but for our current purposes it will be enough to understand the theorem and to apply it in subsequent chapters.

If we want to talk colloquially, we have actually already seen the Central Limit Theorem. We noted previously that “most histograms are (more or less) bell-shaped,” which is in fact one way to state the Central Limit Theorem. To state this theorem precisely, we need to specify, among other things, exactly which normal distribution we are talking about.

This theorem is perhaps somewhat hard to understand, so here is a more colloquial restatement.

The importance of this theorem is that it allows us to start with an arbitrary and possibly unknown distribution, yet use the normal distribution with appropriate mean and standard deviation to perform various computations, at least approximately.

If we roll a single die, the numbers 1 to 6 are all equally likely to come up. Thus, the probability for each outcome is 1/6 so that the distribution looks like Figure 4.16.

Figure 4.16 Uniform distribution for tossing one die

Table 4.4 Average for tossing two dice

	1	2	3	4	5	6
1	2/2	3/2	4/2	5/2	6/2	7/2
2	3/2	4/2	5/2	6/2	7/2	8/2
3	4/2	5/2	6/2	7/2	8/2	9/2
4	5/2	6/2	7/2	8/2	9/2	10/2
5	6/2	7/2	8/2	9/2	10/2	11/2
6	7/2	8/2	9/2	10/2	11/2	12/2

Note that the mean m = 3.5 and the standard deviation s = 1.7078.

If we throw two dice and record their average, we get the outcomes 2/2, 3/2, 4/2, 5/2, 6/2, 7/2, 8/2, 9/2, 10/2, 11/2, and 12/2. We can list these outcomes in Table 4.4, similar to what we did before.

As before, we can determine the probabilities by counting to create the distribution in Figure 4.17.

Now we throw three dice and record the average. There are 6 × 6 × 6 = 216 total possibilities, with probabilities like P(average = 3/3) = P({1,1,1}) = 1/216, P(average = 4/3) = P({1,1,2}, {1,2,1}, {2,1,1}) = 3/216, and so on. We could show the outcomes in a three-dimensional table, but instead we simply show the resulting distribution in Figure 4.18. While we are at it, we also show the result of recording the average of four dice.

We can see that as the sample size N increases, the distribution looks more and more bell-shaped (normal), exactly as the Central Limit Theorem predicts.

The Central Limit Theorem Applet

If you want to see the Central Limit Theorem in action, check out the Central Limit Applet (see Figure 4.19; it requires the latest version of the Java plug-in, which you can download for free).

Try the following:

• Click on the preceding link for the Central Limit Theorem applet.
• Click on the “Start CLT Applet” button (the applet might take a few seconds to initialize).

Figure 4.17 Distribution for tossing two dice

Figure 4.18 Distribution for tossing three and four dice

Figure 4.19 Central Limit Theorem Applet

Source: http://www.mathcs.org/java/programs/CLT/clt.html

○When you click “Start,” the program will pick a random sample from a population, compute the mean, and mark where that mean is on the x-axis to start a frequency distribution for the sample mean.

○Then the program picks another random sample, computes its mean, marks it in blue, and continues in that fashion—check the “Slow Motion” checkbox to see what the program does in slow motion.

• After the program is running for a while, notice that the blue bars are slowly building up to a real frequency distribution (the yellow bars underneath show the distribution of the underlying population from which the random samples are selected).

Now try the following:

• Let the program run (at regular speed) for a while. What shape is the distribution of the random samples (blue bars), at least approximately?
• Experiment with different distributions (click on [Pick] to choose another distribution). What shape does the distribution of the sample means (blue chart) have when you pick other distributions for the population? Is that true regardless of the underlying population distribution (yellow chart)?
• What is the mean for the distribution of the sample means (blue chart) in relation to the mean of the distribution of the original distribution (yellow chart)? The figures for the sample means are shown in the category “Sample Stats,” but make sure to run the program for a while before looking at the numbers. Note that these numbers represent the “sample mean” for the distribution of all sample means, and the “sample standard deviation” for the distribution of all sample means (yes, it sounds odd, but that is what it is).
• Is there a relation between the standard deviation of the sample means (blue chart) and that of the original population (yellow chart)? Experiment with sample sizes 16, 25, 36, 49, and 64 to find the relation, but make sure to press the Reset button before using new parameters or sample sizes, and let the program run for a while before estimating the sample stats.

If you have done everything correctly, you have just discovered the Central Limit Theorem! Relax: if you have any trouble with that applet, or if you are not exactly sure what it shows and how it works, do not worry. In this class we are interested in the consequences of the Central Limit Theorem, coming up in the next chapter, and not in that theorem in and of itself.

Proportions and the Binomial Distribution

Of the continuous distributions the normal ones are the most important, but they require a numerical variable. Is there anything we can do for categorical variables? It turns out that if our data is such that it falls into exactly two categories, it can be modeled by a binomial distribution.

Note: A random experiment with exactly two outcomes where the probability of success does not change is sometimes called a Bernoulli Trial, named after the well-known Swiss mathematician Jacob Bernoulli (1654–1705). As so often, the preceding definition sounds really complicated but once you see examples it will become much clearer.

We can describe each of these situations using our terminology for a binomial distribution. In the first case of flipping a single coin, we (arbitrarily) consider heads to be a success (this would be our Bernoulli trial). Then the probability of success is . Since we repeat this experiment 33 times, the random variable x counting the number of successes is B(1/2, 33).

In the second case we consider it a success to contract the disease (which may sound odd). Then p = 0.7 and since we repeat this 100 times, we have a B(0.7, 100) distribution. We could just as well consider it a success not to contract the disease (avoiding a disease does sound more successful). In that case p = 1 − 0.7 = 0.3 and this variable, call it y, counting the number of successes, is B(0.3, 100).

For the last case we consider a vote for candidate A a success so that p = 0.4. Since we have 25,000 voters, our distribution is B(0.4, 25,000).

It turns out that there is a (relatively) simple formula for a binomial distribution but it requires the formula , where and 0! = 1. We pronounce n! as “n factorial” and as “n choose k” or “choose k out of n.” For example and Note that “n choose k” always comes out an integer even though at first glance that seems unlikely.

This is pretty nifty. It gives us the probability of getting k successes in N total tries, each of which has probability of success p.

Some of the probabilities are easy to determine. For example, the probability of obtaining six heads should clearly be (0.5)6. Also, the probability of obtaining no heads is equal to the probability of getting six tails, which again is (0.5)6. For the probabilities in between we need to apply the preceding formula; see Table 4.5 and Figure 4.20 for the results.

Table 4.5 Binomial distribution B(1/2, 6)

k	P(X = k)
0
1
2
3
4
5
6

Figure 4.20 Binomial distribution B(1/2, 6)

Using Excel to Compute the Binomial Distribution

Excel, of course, includes functions to easily compute P(x = k) if x is B(p, N).

FACT(n)	computes n!
COMBIN(n, k)	computes “n choose k,” that is, the number of ways to select k objects from n objects
BINOMDIST(x, N, p, false)	computes probability of obtaining k successes in N trials if the probability of success is p

Using Excel’s BINOM.DIST (or BINOMDIST in Excel 2007) function it is easy to create the probability distribution for B(0.2, 40); see Figure 4.21. We can then create the cumulative probability chart to determine Q1 = 7, Q3 = 11, and the median = 9, as explained in Chapter 3. The mean of B(0.2, 40) is 8. The distribution in Figure 4.21 looks approximately normal but shifted slightly to the right.

Figure 4.21 Distribution for B(0.2, 40)

Let us conclude this chapter with an interesting application that might give you pause for thought.

Let x be a variable that counts the number of successful shuttle launches. Because of our assumptions, x has a binomial distribution with B(0.992, N). Thus, the probability of 25 successful launches (and no failures) is or about 82 percent. The probability of 135 successes in a row is similarly or only 33 percent.

To find N such that the probability of N successful launches drops below 50 percent we need to solve . We can take the natural logarithm on both sides to find that . Thus, the probability of 87 successful launches has dropped to less than 50 percent. However, the probability of the 88th launch is again 99.2 percent, regardless of what happened before. Such is the nature of the binomial distribution: no matter how close the probability of success p is to 1, the probability of N successes in a row eventually drops to zero! Still, each trial has again a probability of success p, regardless of how many successes in a row already happened. To put this in words: in a binomial distribution it is certain that disaster will strike eventually, but you cannot predict when.

Excel Demonstration

Recall that to solve a probability problem using a binomial distribution, you would need the number of successes you are looking for, the number of trials, and the probability of success. In the following text is a quick tip sheet on the scripts to use in Excel for specific situations. Note that TRUE/FALSE represents whether or not we are looking for the cumulative probability. In other words, if we are looking for the probability of 5, and if we were to say TRUE in our script, that means we would get the probability of everything up to and including 5 (the cumulative amount). If we say FALSE (meaning we are saying NO to the cumulative) that means we only want 5, not the cumulative value.

Rules for Finding Binomial Probabilities in Excel

• If the question asks you to find the probability of exactly one number, use BINOMDIST(successes,trials,probability,FALSE).
• If the question asks you to find the probability of up to and including a number (less than or equal to a number), use BINOMDIST(successes,trials,probability,TRUE).
• If the question asks you to find the probability of less than a number, use BINOMDIST(successes,trials,probability,TRUE) − BINOMDIST(successes,trials,probability,FALSE).
• If the question asks you to find the probability of at least one number (greater than or equal to a number), use (1 − BINOMDIST(successes,trials,probability,TRUE)) + BINOMDIST(successes,trials,probability,FALSE).
• If the question asks you to find the probability of greater than a number, use 1 − BINOMDIST(successes,trials,probability,TRUE).

The key parameters to this problem are: sample size (trials) N = 3, probability of “success” p = 5 percent, and number of “successes” (a) exactly 0, (b) 1 or more, or (c) more than one. To solve in Excel, we would use the following formulas:

a.BINOMDIST(0,3,0.05,FALSE) to get the result of: 0.857375 or around 86 percent

b.1 − BINOMDIST(0,3,0.05,FALSE) to get the result of: 0.142625 or around 14 percent

c.1 − BINOMDIST(1,3,0.05,TRUE) to get the result of: 0.142625 or around 14 percent

The key parameters for this problem are: sample size (trials) N = 5, probability of success p = 70 percent, and number of successes (a) exactly 5, (b) 3 or more, or (c) fewer than two. To solve this in Excel, we would use the following formulas:

a.=BINOMDIST(5,5,0.7,FALSE) to get the result of: 0.16807 or around 17 percent

b.=1 − BINOMDIST(3,5,0.7,TRUE) + BINOMDIST(3,5,0.7,FALSE) to get the result of 0.83692 or around 84 percent

c.=BINOMDIST(2,5,0.7,TRUE) − BINOMDIST(2,5,0.7,FALSE) to get the result of 0.03078 or about 3 percent

Solving Probability Problems Using Normal Distribution Techniques

Company P determined that its truck drivers making deliveries are spending a lot of time on the road, and the trucks are becoming worn out quickly. Company P determined that the annual distance traveled per truck is normally distributed, with a mean of 72,000 miles and a standard deviation of 17,000 miles.

a.What proportion of trucks can be expected to travel between 38,000 and 62,000 miles in the year?

b.What percentage of the trucks travel less than 35,000 miles in the year?

c.What percentage of the trucks travel more than 57,000 miles in the year?

d.How many miles will be traveled by more than 72 percent of the trucks?

In Excel 2013, go to Formulas, and click Insert Function (which is the fx entry in the input line). Type in NORMDIST and select GO. Double-click on the NORMDIST option (see Figure 4.22).

To solve question A of finding the probability of traveling between 38,000 and 62,000, find the probability of 62,000 and then subtract from the probability of 3800,000:

• To find the probability of 62,000, input 62,000 for X, 72,000 for MEAN, 17,000 for Standard Deviation, and TRUE for Cumulative. The probability for 62,000 is 0.27 (see Figure 4.23).
• Now change X to 38,000 and the probability is 0.02.
• Subtract 0.27 and 0.02 to get 0.25 or 25 percent as the answer to A.

Figure 4.22 Dialog for NORMDIST function

Figure 4.23 The standard normal probability for x = 62,000

To solve question B, leave the same dialog box up and leave the mean, standard deviation, and cumulative value as they were. Replace X with 35,000 to get 0.01 or 1 percent for B.

To solve question C, find the probability of less than or equal to 57,000 first by replacing X with 57,000 and leaving cumulative as True. This gives you 0.18 or 18 percent as the probability of getting less than or equal to 57,000. To find the probability of getting more than 55,000, subtract 0.18 from 1 to get 0.82 or 82 percent, which is the solution.

To solve question D, we must use the inverse function of NORMDIST since we want to compute a cut-off value for x to get a specified probability. Go to Insert Function and type in NORMINV (see Figure 4.24).

You are asked to find how many miles will be traveled by more than 72 percent of the trucks, so you are looking for the instances that fall above 72 percent, which is the top 27 percent. If it had asked us for less than 72 percent of the trucks then we would have been concerned with those trucks in the 0 to 72 percent range, not the 73 to 100 percent range. Input the following numbers into the dialog box shown in Figure 4.24.

Figure 4.24 Dialog for NORMINV function

As you can see, the answer to D is 61,582 miles.