Question: In Example, Janet's handling of the fact that her data showed 26 volunteers in the control group when there should have been only 25.
Example: Janet's (Hypothetical) Dissertation Research This is a hypothetical example to illustrate some of the subtle (and not so subtle) ways experimenter bias can alter the data collected for a study. Janet is a Ph.D. student and is under tremendous pressure to complete her research successfully. For her study, she hypothesized that role-playing assertiveness training for women would help them learn to say "no" to telephone solicitors. She recruited 50 undergraduate women as volunteers for the study. The plan was that each volunteer would come to her office for half an hour. For 25 of the volunteers (the control group) she would simply talk with them for 30 minutes about a variety of topics, including their feelings about saying "no" to unwanted requests. For the other 25 volunteers Janet would spend 15 minutes on similar discussion and the remaining 15 minutes on a prespecified role-playing scenario in which the volunteer got to practice saying "no" in various situations.
Two weeks after each volunteer's visit, Brad, a colleague of Janet's, would phone them anonymously, pretending to be a telephone solicitor selling a magazine for a good price, and would record the conversation so that Janet could determine whether or not they were able to say "no." It's the first day of the experiment and the first volunteer is in Janet's office. Janet has the randomization list that was prepared by someone else, which randomly assigns each of the 50 volunteers to either Group 1 or Group 2. This first volunteer appears to be particularly timid, and Janet is sure she won't be able to learn to say "no" to anyone. The randomization list says that volunteer 1 is to be in Group 2. But what was Group 2? Did she say in advance? She can't remember. Oh well, Group 2 will be the control group. The next volunteer comes in and, according to the randomization list, volunteer 2 is to be assigned to Group 1, which now is defined to be the role-playing group. Janet follows her predefined protocol for the half hour.
But when the half hour is over, the student doesn't want to leave. Just then Brad comes by to say "hello," and the three of them spend another half hour having an amiable conversation. The second phase of the experiment begins and Brad begins phoning the volunteers. The conversations are recorded so that Janet can assess the results. When listening to volunteer 2's conversation, Janet notices that almost immediately she says to Brad, "Your voice sounds awfully familiar, do I know you?" When he assures her that she does not and asks her to buy the magazine, she says, "I can't place my finger on it, but this is a trick, right? I'm sure I know your voice. No thanks, no magazine!" Janet records the data: a successful "no" to the solicitation. Janet listens to another call, and although she is supposed to be blind to which group the person was in, she recognizes the voice as being one of the role-playing volunteers. Brad pitches the magazine to her and her response is "Oh, I already get that magazine.
But if you are selling any others, I might be able to buy one." Janet records the data: a successful "no" to the question of whether she wants to buy that magazine. The second phase is finally over and Janet has a list of the results. But now she notices a problem. There are 26 people listed in the control group and 24 listed in the roleplaying group. She tries to resolve the discrepancy but can't figure it out. She notices that the last two people on the list are both in the control group and that they both said This example illustrates just a few of the many ways in which experimenter bias can enter into a research study. Even when it appears that protocols are carefully in place, in the real world it is nearly impossible to place controls on all aspects of the research. In this case, notice that every decision Janet made benefited her desired conclusion that the role-playing group would learn to say "no." It is up to the researchers to take utmost care not to allow these kinds of unethical influences on the results.