2.2 Revising Probabilities with Bayes’ Theorem
Bayes’ Theorem is used to incorporate additional information as it is made available and help create revised or posterior probabilities from the original or prior probabilities. This means that we can take new or recent data and then revise and improve upon our old probability estimates for an event (see Figure 2.3). Let us consider the following example.
Figure 2.3 Using Bayes’ Process
A cup contains two dice identical in appearance. One, however, is fair (unbiased) and the other is loaded (biased). The probability of rolling a 3 on the fair die is or 0.166. The probability of tossing the same number on the loaded die is 0.60.
We have no idea which die is which but select one by chance and toss it. The result is a 3. Given this additional piece of information, can we find the (revised) probability that the die rolled was fair? Can we determine the probability that it was the loaded die that was rolled?
The answer to these questions is yes, and we do so by using the formula for joint probability under statistical dependence and Bayes’ Theorem. First, we take stock of the information and probabilities available. We know, for example, that since we randomly selected the die to roll, the probability of it being fair or loaded is 0.50:
We also know that
Next, we compute joint probabilities P(3 and fair) and P(3 and loaded) using the formula
A 3 can occur in combination with the state “fair die” or in combination with the state “loaded die.” The sum of their probabilities gives the unconditional or marginal probability of a 3 on the toss; namely,
If a 3 does occur, and if we do not know which die it came from, the probability that the die rolled was the fair one is
The probability that the die rolled was loaded is
These two conditional probabilities are called the revised or posterior probabilities for the next roll of the die.
Before the die was rolled in the preceding example, the best we could say was that there was a 50–50 chance that it was fair (0.50 probability) and a 50–50 chance that it was loaded. After one roll of the die, however, we are able to revise our prior probability estimates. The new posterior estimate is that there is a 0.78 probability that the die rolled was loaded and only a 0.22 probability that it was not.
Using a table is often helpful in performing the calculations associated with Bayes’ Theorem. Table 2.3 provides the general layout for this, and Table 2.4 provides this specific example.
Table 2.3 Tabular Form of Bayes’ Calculations Given That Event B Has Occurred
| STATE of Nature | P(B | STATE OF NATURE) | PRIOR PROBABILITY | JOINT PROBABILITY | POSTERIOR PROBABILITY |
|---|---|---|---|---|
| A | P(B A) | P(A) | P(B and A) | P(B and A) P(B) P(A B) |
| P(B A ) | P(A ) | P(B and A ) P(B) P(A B) |
Table 2.4 Bayes’ Calculations Given That a 3 Is Rolled in This Example
| STATE OF NATURE | P(3 | STATE OF NATURE) | PRIOR PROBABILITY | JOINT PROBABILITY | POSTERIOR PROBABILITY |
|---|---|---|---|---|
| Fair die | 0.166 | 0.5 | ||
| Loaded die | 0.600 | 0.5 | ||
General Form of Bayes’ Theorem
Revised probabilities can also be computed in a more direct way using a general form for Bayes’ Theorem:
where
We originally saw in Equation 2-3 the conditional probability of event A, given event B, is
Thomas Bayes derived his theorem from this. Appendix 2.1 shows the mathematical steps leading to Equation 2-5. Now let’s return to the example.
Although it may not be obvious to you at first glance, we used this basic equation to compute the revised probabilities. For example, if we want the probability that the fair die was rolled, given the first toss was a 3—namely, P(fair die rolled)—we can let
We can then rewrite Equation 2-5 and solve as follows:
This is the same answer that we computed earlier. Can you use this alternative approach to show that Either method is perfectly acceptable, but when we deal with probability revisions again in Chapter 3, we may find that Equation 2-5 or the tabular approach is easier to apply.