A decision tree ID3 algorithm first constructs a decision tree from the input data and then classifies a new data instance using this constructed tree. A decision tree is constructed by selecting the attribute for branching with the highest information gain. The information gain measures how much information can be learned in terms of the gain in the information entropy.

The decision tree algorithm can achieve a different result from other algorithms such as Naive Bayes' algorithm. In the next chapter, we will learn how to combine various algorithms or classifiers into a decision forest (called random forest) in order to achieve a more accurate result.