1. How narrowly is the research problem defined"? In your opinion, is it too narrow? Is it too broad? Explain.
2. Was the research setting artificial (e.g., a laboratory setting)? If yes, do you think that the gain in the control of extraneous variables offsets the potential loss of information that would be obtained in a study in a more real-life setting? Explain.
3. Are there any obvious flaws or weaknesses in the researcher's methods of observa¬tion? Explain. (Note: Observation or measurement is often described under the sub-heading "Instrumentation.")
4. Are there any obvious sampling flaws? Explain.
5. Was the analysis statistical or nonstatistical? Was the description of the results easy to understand? Explain.
6. Were the descriptions of procedures and methods of observation sufficiently detailed? Were any important details missing? Explain.
7. Does the report lack information on matters that are potentially important for evaluat¬ing it?
8. Overall, was the research obviously very weak? If yes, briefly describe its weaknesses and speculate on why it was published despite them.
9. Does the researcher describe related theories?
10. Does the researcher imply that his or her research proves something? Do you believe that it proves something? Explain.
11. Do you think that as a result of reading this chapter and evaluating a research report you are becoming more expert at evaluating research reports? Explain.