Avoiding Over fitting - Artificial intelligence
As we discussed in the last lecture, over fitting is a normal problem in machine learning. Decision trees suffer from that, because they are skilled to stop when they have completely classify all the training data, i.e., every branch is complete just far sufficient to suitably categories the examples important to the branch. Various ways to overcoming over fitting in decision trees have been used. As summarized by Tom Mitchell, these attempts fit into two types:
· Stop growing the tree before it reaches excellence.
- Allow the tree to complete grow, and then post-prune some of the branches from it.
The second approach has been found to be more useable in practice. Both methods boil down to the question of shaping the right tree size. See Chapter 3 of Tom Mitchell's book for more detailed explanation of over fitting prevention in decision tree knowledge.