![]() So the information Gain for a given attribute is computed by taking the entropy of the whole set and subtracting it with the entropies of sets that are obtained by breaking the whole set into one piece per attribute category. What your decision tree tries to achieve is to reduce the impurity of the whole set. Well, first you calculate the entropy of the whole set. ![]() How does a decision tree use the entropy? If one color is dominant then the entropy will be close to 0, if the colors are very mixed up, then it is close to the maximum (2 in your case). Then your entropy is between the two values. The maximum entropy of a 4 class set is 2. if you had 12 balls and 3 balls of each color. In your case, you have 4 different class labels, so the maximum entropy would be e.g. If you have just two, the maximum entropy is 1. The highest possible entropy value depends on the number of class labels.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |