Experiments, trials, events, and sample spaces

Statisticians and others who work with probability refer to a process like tossing a coin or throwing a die as an experiment. Each time you go through the process, that’s a trial.

This might not fit your personal definition of an experiment (or of a trial, for that matter), but for a statistician, an experiment is any process that produces one of at least two distinct results (like heads or tails).

Another piece of the definition of an experiment: You can’t predict the result with certainty. Each distinct result is called an elementary outcome. Put a bunch of elementary outcomes together and you have an event. For example, with a die, the elementary outcomes 2, 4, and 6 make up the event “even number.”

Put all the possible elementary outcomes together and you’ve got yourself a sample space. The numbers 1, 2, 3, 4, 5, and 6 make up the sample space for a die. “Heads” and “tails” make up the sample space for a coin.

Sample spaces and probability

How does all this play into probability? If each elementary outcome in a sample space is equally likely, the probability of an event is

So the probability of tossing a die and getting an even number is

If the elementary outcomes are not equally likely, you find the probability of an event in a different way. First, you have to have some way of assigning a probability to each one. Then you add up the probabilities of the elementary outcomes that make up the event.

A couple of things to bear in mind about outcome probabilities: Each probability has to be between zero and one. All the probabilities of elementary outcomes in a sample space have to add up to 1.00.

How do you assign those probabilities? Sometimes you have advance information — such as knowing that a coin is biased toward coming up heads 60 percent of the time. Sometimes you just have to think through the situation to figure out the probability of an outcome.

Here’s a quick example of “thinking through.” Suppose a die is biased so that the probability of an outcome is proportional to the numerical label of the outcome: A 6 comes up six times as often as a 1, a 5 comes up five times as often as a 1, and so on. What is the probability of each outcome? All the probabilities have to add up to 1.00, and all the numbers on a die add up to 21 (1+2+3+4+5+6 = 21), so the probabilities are: pr(1) = 1/21, pr(2) = 2/21, … , pr(6) = 6/21.

Union and intersection

On a toss of a fair die, what’s the probability of rolling a 1 or a 4? Mathematicians have a symbol for or. It’s called union, and it looks like this: ∪. Using this symbol, the probability of a 1 or a 4 is pr(1 ∪ 4).

In approaching this kind of probability, it’s helpful to keep track of the elementary outcomes. One elementary outcome is in each event, so the event “1 or 4” has two elementary outcomes. With a sample space of six outcomes, the probability is 2/6 or 1/3. Another way to calculate this is

Here’s a slightly more involved one: What’s the probability of rolling a number between 1 and 3 or a number between 2 and 4?

Just adding the elementary outcomes in each event won’t get it done this time. Three outcomes are in the event “between 1 and 3,” and three are in the event “between 2 and 4.” The probability can’t be 3 + 3 divided by the six outcomes in the sample space, because that’s 1.00, leaving nothing for pr(5) and pr(6). For the same reason, you can’t just add the probabilities.

The challenge arises in the overlap of the two events. The elementary outcomes in “between 1 and 3” are 1, 2, and 3. The elementary outcomes in “between 2 and 4” are 2, 3, and 4. Two outcomes overlap: 2 and 3. In order to not count them twice, the trick is to subtract them from the total.

A couple of things will make life easier as I proceed. I abbreviate “between 1 and 3” as A and “between 2 and 4” as B. Also, I use the mathematical symbol for “overlap.” The symbol is ∩ and it’s called intersection.

Using the symbols, the probability of “between 1 and 3” or “between 2 and 4” is

You can also work with the probabilities:

The general formula is

Why was it okay to just add the probabilities together in the earlier example? Because pr(1 ∩ 4) is zero: It’s impossible to roll a 1 and a 4 in the same toss of a die. Whenever pr(A ∩ B) = 0, A and B are said to be mutually exclusive.

Intersection again

Imagine throwing a coin and rolling a die at the same time. These two experiments are independent because the result of one has no influence on the result of the other.

What’s the probability of getting Heads and a 4? You use the intersection symbol and write this as pr(Heads ∩ 4)

Start with the sample space. Table 18-1 lists all elementary outcomes.

As the table shows, 12 outcomes are possible. How many outcomes are in the event “Heads and 4”? Just one. So

You can also work with the probabilities:

In general, if A and B are independent,

Working with the probabilities

In general, if you have two events A and B,

as long as pr(B) isn’t zero.

For the intersection in the numerator on the right, this is not a case where you just multiply probabilities together. In fact, if you could do that, you wouldn’t have a conditional probability, because that would mean A and B are independent. If they’re independent, one event can’t be conditional on the other.

You have to think through the probability of the intersection. In a die, how many outcomes are in the event “5 ∩ Greater than 2”? Just one, so pr(5 ∩ Greater than 2) is 1/6, and

The foundation of hypothesis testing

All the hypothesis testing I show you in previous chapters involves conditional probability. When you calculate a sample statistic, compute a statistical test, and then compare the test statistic against a critical value, you’re looking for a conditional probability. Specifically, you’re trying to find

If that conditional probability is low (less than .05 in all the examples I show you in hypothesis-testing chapters), you reject H₀.

Permutations

Suppose you have to arrange five objects into a sequence. How many ways can you do that? For the first position in the sequence, you have five choices. After you make that choice, you have four choices for the second position. Then you have three choices for the third, two for the fourth, and one for the fifth. The number of ways is (5)(4)(3)(2)(1) = 120.

In general, the number of sequences of N objects is N(N-1)(N-2) … (2)(1). This kind of computation occurs fairly frequently in probability world, and it has its own notation, N! You don’t read this by screaming out “N” in a loud voice. Instead, it’s “N factorial.” By definition, 1! = 1, and 0! = 1.

Now for the good stuff. If you have to order the 26 letters of the alphabet, the number of possible sequences is 26!, a huge number. But suppose the task is to create 5-letter sequences so that no letter repeats in the sequence. How many ways can you do that? You have 26 choices for the first letter, 25 for the second, 24 for the third, 23 for the fourth, 22 for the fifth, and that’s it. So that’s (26)(25)(24)(23)(22). Here’s how that product is related to 26!:

Each sequence is called a permutation. In general, if you take permutations of N things r at a time, the notation is _NP_r (the P stands for permutation). The formula is

Just for completeness, here’s another wrinkle. Suppose that I allow repetitions in these sequences of 5. That is, aabbc is a permissible sequence. In that case, the number of sequences is 26 × 26 × 26 × 26 × 26, or as mathematicians would say, “26 raised to the fifth power.” Or as mathematicians would write, “26⁵.”

Combinations

In the preceding example, these sequences are different from one another: abcde, adbce, dbcae, and on and on and on. In fact, you could come up with 5! = 120 of these different sequences just for the letters a, b, c, d, and e.

Suppose that I add the restriction that one of these sequences is no different from another, and all I’m concerned about is having sets of five nonrepeating letters in no particular order. Each set is called a combination. For this example, the number of combinations is the number of permutations divided by 5!:

In general, the notation for combinations of N things taken r at a time is _NC_r (the C stands for combination). The formula is

Now for that completeness wrinkle again. Suppose that I allow repetitions in these sequences. How many sequences would I have? It turns out to be equivalent to N+r-1 things taken N-1 at a time, or _N+r+1C_N-1. For this example, that would be ₃₀C₂₅.

FACT

FACT, which computes factorials, is surprisingly not categorized as Statistical. Instead, you’ll find it on the Math & Trig Functions menu. It’s easy to use. Supply it with a number, and it returns the factorial. Here are the steps:

1. Select a cell for FACT's answer.
2. From the Math & Trig Functions menu, select FACT to open its Function Arguments dialog box.

3. In the Function Arguments dialog box, enter the appropriate value for the argument.

   In the Number box, I typed the number whose factorial I want to compute.

   The answer appears in the dialog box. If I enter 5, for example, 120 appears.

4. Click OK to put the answer into the selected cell.

PERMUT and PERMUTIONA

You find these two on the Statistical Functions menu. As its name suggests, PERMUT enables you to calculate _NP_r. Here’s how to use it to find ₂₆P₅, the number of 5-letter sequences (no repeating letters) that you can create from the 26 letters of the alphabet. In a permutation, remember, abcde is considered different from bcdae. Follow these steps:

1. Select a cell for PERMUT's answer.
2. From the Statistical Functions menu, select PERMUT to open its Function Arguments dialog box. (See Figure 18-1.)

3. In the Function Arguments dialog box, type the appropriate values for the arguments.

   In the Number box, I entered the N in _NP_r. For this example, N is 26.

   In the Number_chosen box, I entered the r in _NP_r. That would be 5.

   With values entered for both arguments, the answer appears in the dialog box. For this example, the answer is 7893600.

4. Click OK to put the answer into the selected cell.

PERMUTIONA does the same thing, but with repetitions allowed. Its Function Arguments dialog box looks exactly like the one for PERMUT. Its answer is equivalent to N^r. For this example, by the way, that answer is 1181376.

COMBIN and COMBINA

COMBIN works pretty much the same way as PERMUT. Excel categorizes COMBIN and COMBINA as Math & Trig functions.

Here’s how you use them to find ₂₆C₅, the number of ways to construct a 5-letter sequence (no repeating letters) from the 26 letters of the alphabet. In a combination, abcde is considered equivalent to bcdae.

1. Select a cell for COMBIN’s answer.
2. From the Math & Trig Functions menu, select COMBIN to open its Function Arguments dialog box.

3. In the Function Arguments dialog box, type the appropriate values for the arguments.

   In the Number box, I entered the N in _NC_r. Once again, N is 26.

   In the Number_chosen box, I entered the r in _NC_r. And again, r is 5.

   With values entered for both arguments, the answer appears in the dialog box. For this example, the answer is 65870.

4. Click OK to put the answer into the selected cell.

If you allow repetitions, use COMBINA. Its Function Arguments dialog box looks just like COMBIN’s. For this example, its answer is equivalent to ₃₀C₂₅ (142506).

BINOM.DIST and BINOM.DIST.RANGE

These are Excel’s worksheet functions for the binomial distribution. Use BINOM.DIST to calculate the probability of getting four 3’s in ten tosses of a fair die:

1. Select a cell for BINOM.DIST’s answer.
2. From the Statistical Functions menu, select BINOM.DIST to open its Function Arguments dialog box. (See Figure 18-4.)

3. In the Function Arguments dialog box, type the appropriate values for the arguments.

   In the Number_s box, I entered the number of successes. For this example, the number of successes is 4.

   In the Trials box, I entered the number of trials. The number of trials is 10.

   In the Probability_s box, I entered the probability of a success. I entered 1/6, the probability of a 3 on a toss of a fair die.

   In the Cumulative box, one possibility is FALSE for the probability of exactly the number of successes entered in the Number_s box. The other is TRUE for the probability of getting that number of successes or fewer. I entered FALSE.

   With values entered for all the arguments, the answer appears in the dialog box.

4. Click OK to put the answer into the selected cell.

To give you a better idea of what the binomial distribution looks like, I use BINOM.DIST (with FALSE entered in the Cumulative box) to find pr(0) through pr(10), and then I use Excel’s graphics capabilities (refer to Chapter 3) to graph the results. Figure 18-5 shows the data and the graph.

Incidentally, if you type TRUE in the Cumulative box, the result is .984 (and some more decimal places), which is pr(0) + pr(1) + pr(2) + pr(3) + pr(4).

Figure 18-5 is helpful if you want to find the probability of getting between four and six successes in ten trials. Find pr(4), pr(5), and pr(6) and add the probabilities.

A much easier way, especially if you don’t have a chart like Figure 18-5 handy or if you don’t want to apply BINOM.DIST three times, is to use BINOM.DIST.RANGE. Figure 18-6 shows the dialog box for this function, supplied with values for the arguments. After all the arguments are entered, the answer (0.069460321) appears in the dialog box.

If you don’t put a value in the Number_s2 box, BINOM.DIST.RANGE returns the probability of whatever you entered into the Number_s box. If you don’t put a value in the Number_s box, the function returns the probability of, at most, the number of successes in the Number_s2 box (for example, the cumulative probability).

NEGBINOM.DIST

As its name suggests, NEGBINOM.DIST handles the negative binomial distribution. I use it here to work out the earlier example — the probability of getting five failures (tosses that result in anything but a 3) before the fourth success (the fourth 3). Here are the steps:

1. Select a cell for NEGBINOM.DIST's answer.
2. From the Statistical Functions menu, select NEGBINOM.DIST to open its Function Arguments dialog box. (See Figure 18-7.)

3. In the Function Arguments dialog box, type the appropriate values for the arguments.

   In the Number_f box, I entered the number of failures. The number of failures is 5 for this example.

   In the Number_s box, I entered the number of successes. For this example, that’s 4.

   In the Probability_s box, I entered 1/6, the probability of a success.

   In the Cumulative box, I entered FALSE. This gives the probability of the number of successes. If I enter TRUE, the result is the probability of at most that number of successes.

   With values entered for all the arguments, the answer appears in the dialog box. The answer is 0.017 and some additional decimal places.

4. Click OK to put the answer into the selected cell.

BINOM.INV

This function is tailor-made for binomial-based hypothesis testing. Give BINOM.INV the number of trials, the probability of a success, and a criterion cumulative probability. BINOM.INV returns the smallest value of x (the number of successes) for which the cumulative probability is greater than or equal to the criterion.

Here are the steps for the hypothesis testing example I just showed you:

1. Select a cell for BINOM.INV’s answer.
2. From the Statistical Functions menu, select BINOM.INV and click OK to open its Function Arguments dialog box. (See Figure 18-8.)

3. In the Function Arguments dialog box, enter the appropriate values for the arguments.

   In the Trials box, I entered 10, the number of trials.

   In the Probability_s box, I entered the probability of a success. In this example it’s 1/6, the value of π according to H₀.

   In the Alpha box, I entered the cumulative probability to exceed. I entered .95 because I want to find the critical value that cuts off the upper 5 percent of the binomial distribution.

   With values entered for the arguments, the critical value, 4, appears in the dialog box.

4. Click OK to put the answer into the selected cell.

As it happens, the critical value is the number of successes in the sample. The decision is to reject H₀.

More on hypothesis testing

In some situations, the binomial distribution approximates the standard normal distribution. When this happens, you use the statistics of the normal distribution to answer questions about the binomial distribution.

Those statistics involve z-scores, which means that you have to know the mean and the standard deviation of the binomial. Fortunately, they’re easy to compute. If N is the number of trials and π is the probability of a success, the mean is

the variance is

and the standard deviation is

The binomial approximation to the normal is appropriate when N π ≥ 5 and N(1 – π) ≥ 5.

When you test a hypothesis, you’re making an inference about π, and you have to start with an estimate. You run N trials and get x successes. The estimate is

To create a z-score, you need one more piece of information — the standard error of P. This sounds harder than it is, because this standard error is just

Now you’re ready for a hypothesis test.

Here’s an example. The CEO of FarKlempt Robotics, Inc., believes that 50 percent of FarKlempt robots are purchased for home use. A sample of 1,000 FarKlempt customers indicates that 550 of them use their robots at home. Is this significantly different from what the CEO believes? The hypotheses:

H₀: π = .50

H₁: π ≠ .50

I set α = .05

N π = 500, and N(1 - π) = 500, so the normal approximation is appropriate.

First, calculate P:

Now create a z-score:

With α = .05, is 3.162 a large enough z-score to reject H₀? An easy way to find out is to use the worksheet function NORM.S.DIST. (See Chapter 8.) If you do, you’ll find that this z-score cuts off less than .01 of the area in the upper tail of the standard normal distribution. The decision is to reject H₀.

HYPGEOM.DIST

This function calculates everything for you when you deal with the hypergeometric distribution. Here’s how to use it to work through the preceding example:

1. Select a cell for HYPGEOM.DIST’s answer.
2. From the Statistical Functions menu, select HYPGEOM.DIST to open its Function Arguments dialog box. (See Figure 18-9.)

3. In the Function Arguments dialog box, enter the appropriate values for the arguments.

   In the Sample_s box, I entered the number of successes in the sample. That number is 3 for this example.

   In the Number_sample box, I entered the number of items in the sample. The sample size for this example is 6.

   In the Population_s box, I entered the number of successes in the population. In this example that’s 7, the number of good light bulbs.

   In the Number_pop box, I entered the number of items in the population. The total number of light bulbs is 16, and that’s the population size.

   In the Cumulative box, I entered FALSE. This gives the probability of the number of successes I entered in the Sample_s box. If I enter TRUE, the function returns the probability of, at most, that number of successes (for example, the cumulative probability).

   With values entered for all the arguments, the answer appears in the dialog box. The answer is 0.367 and some additional decimal places.

4. Click OK to put the answer into the selected cell.

As I do with the binomial, I use HYP.GEOM.DIST to calculate pr(0) through pr(6) for this example. Then I use Excel’s graphics capabilities (refer to Chapter 3) to graph the results. Figure 18-10 shows the data and the chart. My objective is to help you visualize and understand the hypergeometric distribution.