Many professions have board certification exams. Part of the certification process for bank examiners involves a "work basket" of tasks that the examinee must complete in a satisfactory fashion in a fixed time period. New work baskets must be constructed for each round of examinations, and much effort is expended to make the workbaskets comparable (in terms of average score) from exam to exam. This year, two new work baskets (A and B) are being evaluated. We have three old work baskets (C, D, and E) to form a basis for comparison. We have ten paid examinees (1 through 6 are certified bank examiners, 7 through 9 are noncertified bank examiners nearing the end of their training, and 10 is a public accountant with no bank examining experience or training) who will each take all five tests. There are five graders who will each grade ten exams. We anticipate differences between the examinees and the graders; our interest is in the exams, which were randomized so that each examinee took each exam and each grader grades two of each exam.
Student Grader
1 2 3 4 5
1 68D 65A 76E 74C 76B
2 68A 77E 84B 65D 75C
3 73C 85B 72D 68E 62A
4 74E 76C 57A 79B 64D
5 80B 71D 76C 59A 68E
6 69D 75E 81B 68A 68C
7 60C 62D 62E 66B 40A
8 70B 55A 62C 57E 40D
9 61E 67C 53A 63D 69B
10 37A 53B 31D 48C 33E