A decision tree is the arrangement of the data in a tree structure where, at each node, data is separated to different branches according to the value of the attribute at the node.

To construct a decision tree, we will use a standard ID3 learning algorithm that chooses an attribute that classifies the data samples in the best possible way to maximize the information gain - a measure based on information entropy.

In this chapter, you will learn:

• What a decision tree is and how to represent data in a decision tree in example Swim preference
• In the section Information theory concepts of information entropy and information gain theoretically first, then practically applying on example Swim preference
• ID3 algorithm constructing a decision tree from the training data and its implementation in Python
• How to classify new data items using the constructed decision tree in example Swim preference
• How to provide an alternative analysis using decision trees to the problem Playing chess from the previous chapter and how the results of two different algorithms applied may differ
• Verifying your understanding at the exercise section when to use and when not to use decision trees as a method of analysis
• How to deal with data inconsistencies during decision tree construction in example Going shopping