A survey is taken of people in the class of favorite flavor of ice cream among a forced choice of chocolate, vanilla, or strawberry. 70% prefer chocolate, 20% prefer vanilla, and 10% prefer strawberry.
a) What is the entropy of this distribution? (Show how you derived the entropy.)
b) How far is this from the maximum amount of entropy that you could observe for a variable that has three values?
c) Consider the following probability distributions:
P(Flavor=| Gender=male) = <.80,.12,.08>
P(Flavor=| Gender=female) = <.67,.22,.11>
P(Flavor=| StudentType=grad) = <.76,.16,.08>
P(Flavor=| StudentType=undergrad) = <.64,.24,.12>
P(Gender=)=<.75,.25>
P(StudentType=)=<.5,.5>
What would an appropriate first decision be for the decision tree algorithm?