Let us calculate the information gain for the six rows in the swim preference example by taking swimming suit as an attribute. Because we are interested whether a given row of data is classified as no or yes for the question whether one should swim, we will use the swim preference to calculate the entropy and information gain. We partition the set S by the attribute swimming suit:

S_none={(none,cold,no),(none,warm,no)}

S_small={(small,cold,no),(small,warm,no)}

S_good={(good,cold,no),(good,warm,yes)}

The information entropy of S is E(S)=-(1/6)*log₂(1/6)-(5/6)*log₂(5/6)~0.65002242164.

The information entropy of the partitions is:

E(S_none)=-(2/2)*log₂(2/2)=-log₂(1)=0 since all instances have the class no.

E(S_small)=0 for a similar reason.

E(S_good)=-(1/2)*log₂(1/2)=1

Therefore, the information gain is:

IG(S,swimming suit)=E(S)-[(2/6)*E(S_none)+(2/6)*E(S_small)+(2/6)*E(S_good)]

=0.65002242164-(1/3)=0.3166890883

If we chose the attribute water temperature to partition the set S, what would be the information gain IG(S,water temperature)? The water temperature partitions the set S into the following sets:

S_cold={(none,cold,no),(small,cold,no),(good,cold,no)}

S_warm={(none,warm,no),(small,warm,no),(good,warm,yes)}

Their entropies are:

E(S_cold)=0 as all instances are classified as no.

E(S_warm)=-(2/3)*log₂(2/3)-(1/3)*log₂(1/3)~0.91829583405

Therefore, the information gain from partitioning the set S by the attribute water temperature is:

IG(S,water temperature)=E(S)-[(1/2)*E(S_cold)+(1/2)*E(S_warm)]

= 0.65002242164-0.5*0.91829583405=0.19087450461

This is less than IG(S,swimming suit). Therefore, we can gain more information about the set S (the classification of its instances) by partitioning it per the attribute swimming suit instead of the attribute water temperature. This finding will be the basis of the ID3 algorithm constructing a decision tree in the next section.