Ensembles

The quote at the beginning of this chapter mentions using ensembles to win machine learning competitions. However, they do have practical applications. I've provided a definition of what ensemble modeling is, but why does it work? To demonstrate this, I've co-opted an example, from the following blog, which goes into depth at a number of ensemble methods:
http://mlwave.com/kaggle-ensembling-guide/.

As I write this chapter, we are only a couple of days away from Super Bowl 51, the Atlanta Falcons versus the New England Patriots. Let's say we want to review our probability of winning a friendly wager where we want to take the Patriots minus the points (3 points as of this writing). Assume that we have been following three expert prognosticators that all have the same probability of predicting that the Patriots will cover the spread (60%). Now, if we favor any one of the so-called experts, it is clear we have a 60% chance to win. However, let's see what creating an ensemble of their predictions can do to increase our chances of profiting and humiliating friends and family.

Start by calculating the probability of each possible outcome for the experts picking New England. If all three pick New England, we have 0.6 x 0.6 x 0.6, or a 21.6% chance, that all three are correct. If any two of the three pick New England then we have (0.6 x 0.6 x 0.3) x 3 for a total of 43.2%. By using majority voting, if at least two of the three pick New England, then our probability of winning becomes almost 65%. 

This is a rather simplistic example but representative nonetheless. In machine learning, it can manifest itself by incorporating the predictions from several average or even weak learners to improve overall accuracy. The diagram that follows shows how this can be accomplished:

In this graphic, we build three different classifiers and use their predicted probabilities as inputs to a fourth and different classifier in order to make predictions on the test data. Let's see how to apply this with R.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset