Problem
The Kappa (κ) statistic compares the classifier result with the ground truth and answers the question of how much better the agreement is (between the ground truth and the machine learning prediction) than would be expected by chance alone. Take the confusion matrix below generated from the naïve Bayes classification model.
a) What is the accuracy of the classifier? How does the accuracy compare to the baseline performance from a ZeroR (zero rule) classifier?
b) Based on the confusion matrix, calculate values for observed agreement, expected agreement, and the kappa score. Comment on the result.