APPENDIX

THE DIVERSITY PREDICTION THEOREM

I present here a more detailed explanation of the diversity prediction theorem. The theorem compares errors. For a numerical prediction, the error equals the square of the difference between the prediction and the true value. Squaring makes all errors positive. A prediction of a sales increase of 10 percent when the true increase equals 7 percent will have an error of nine (9 = (10 − 7)2). A prediction of an increase of 2 percent has an error of twenty-five (25 = (2 − 7)2). If one prediction has an error of nine and the other has an error of twenty-five, the average error equals seventeen. One prediction of 10 percent and one prediction of 2 percent produce an average of 6 percent. The collective error equals the square of the difference between that prediction (six) and the true value (seven). The collective error equals one. The average error (seventeen) exceeds the collective error (one).

The average error equals the average of the predictions’ errors. The collective error of a set of predictions equals the squared error of the average prediction. The crowd makes a better prediction than its average member. This will always be the case with diverse predictions. In other work, I refer to this as the crowd-beats-the-average law.¹ Two effects drive the result: First, overestimates and underestimates cancel out. Second, squaring the differences means the average of two predictions that err in the same direction has lower error than the average error of the two predictions. The error of that average prediction will be less than the average error of the predictions.

We can introduce one more bit of terminology to make an even more precise claim. Define the predictive diversity to equal the variation in a collection of predictions. Variation equals the average of the square of the differences between the individual predictions and the average prediction. In our example, the average prediction equals 6 percent. Each prediction differs from that average by 4 percent, making the squared difference of each prediction to the average equal to sixteen (16 = 42). Therefore, the predictive diversity equals sixteen.

We have computed three numbers: the error of the collective prediction (one), the average error (seventeen), and the predictive diversity (sixteen). Notice that the collective error (one) equals the average error (seventeen) minus the predictive diversity (sixteen). This is the diversity prediction theorem.²

The formal version is written as follows: Assume a collection of N predictors, where Personi denotes the prediction of person i. Let Collective denote the average of the N predictions, and let Truth equal the true value. The following equality holds in all cases:

The theorem does not imply that more diversity is always better.

For an example, suppose that two people (correctly) forecast the amount of snowfall on June 5 in Phoenix, Arizona, at zero inches. Adding a third person who predicts six inches results in a collective prediction of two, and a collective error of four. The new prediction has an error of thirty-six, increasing the average error from zero to twelve. Finally, the diversity increases from zero to eight (the average of four, four, and sixteen). The diversity prediction theorem reads as follows: 4 = 12 − 8. Prior to adding the diverse prediction, it read 0 = 0 − 0.

Suppose that three people predict using an identical model and two other people each use a different model. Suppose also that the three people who use the same model predict with a little more accuracy than the other two. The team of the three best predictors consists of three people who use the same model. The crowd will only be as accurate as each member. If we add the two less accurate predictors and if the gain in diversity outweighs the loss in average accuracy, the diverse group will be more accurate.

To make this more precise, suppose the three people who use the best model have an error equal to ten and that the other two people have errors of fifteen and twenty. The average error of the group of the best equals ten. As that group has no predictive diversity, its collective error also equals ten (10 = 10 − 0). The average error of the diverse group equals fifteen. If the predictive diversity of the second group exceeds five, then the second group will be more accurate than the group of the most accurate individuals.