SELECTING THE WRONG STUDENTS
I helped a university department develop a small decision support system to analyze and rank students who applied to a specialized program. Some of the information was numeric and could easily be processed directly by the system (e.g., grade point average, standardized test scores). Other information required the faculty to make subjective judgments among the students (e.g., extracurricular activities, work experience). The users entered their evaluations of the subjective information via several data analysis screens in which the students were listed in alphabetical order. In order to make the system "easier to use," the reports listing the results of the analysis were also presented in alphabetical order by student name, rather than n order from the highest ranked student to the lowest ranked student. In a series of tests prior to installation, the users selected the wrong students to admit in 20 percent of the cases. They assumed, wrongly, that the students listed first were the highest ranked students and simply selected the first students on the list for admission. Neither the title on the report, nor the fact that all the students' names were in alphabetical order made them realize that they had read the report incorrectly.
QUESTION:
What concerns could this problem raise about the rest of the system?