Suppose that two ergonomists rated 21 different cell phones as to their usability: the ergonomist agrees with the statement that the cell phone is usable (A), the ergonomist is neutral about the usability of the cell phone (N), the ergonomist disagrees with the statement that the cell phone is usable (D). Find a set of ratings for each ergonomist on the 21 cell phones that disagree with each other in each and every case. For example, if Ergonomist 1 agrees that cell phone 1 is usable, then Ergonomist 2 needs to rate cell phone 1 as either neutral or disagree. And so on. Each ergonomist rate 7 phones A, 7 N and 7 D.
b) Compute Kappa to determine the agreement between the two ergonomists. Note that K = (Sa - Sef)/(N - Sef). The parentheses are not in the book, but should be. And note that K can be less than zero.
c) Construct a table with twenty-one rows and two columns. Label the rows 1 -21 (corresponding to the cell phones). Label the columns Ergonomist 1 and Ergonomist
2. Suppose that A (agree) was assigned a value of 1, N (neutral) a value of 2, and D (disagree) a value of 3. Fill in the table with 1s, 2s and 3s consistent with the table you constructed in a).
d) Using the information in the table in c), compute the Pearson product-moment
correlation coefficient to determine the agreement between the two ergonomists.
e) Which measure of agreement would you choose and why?