Map > Data Science > Predicting the Future > Modeling > Classification > Decision Tree > Super Attributes

Decision Tree - Super Attributes

The information gain equation, G(T,X) is biased toward attributes that have a large number of values over attributes that have a smaller number of values. These Super Attributes will easily be selected as the root, resulted in a broad tree that classifies perfectly but performs poorly on unseen instances. We can penalize attributes with large numbers of values by using an alternative method for attribute selection, referred to as Gain Ratio.

The following example shows a frequency table between the target (Play Golf) and the ID attribute which has a unique value for each record of the dataset.

The information gain for ID is maximum (0.94) without using the split information. However, with the adjustment the information gain dropped to 0.25.