The following case study is taken directly from your Lewis, Packard, and Lewis textbook (p.205). It is reproduced here for your convenience.
Evaluation Emergency
Mario Rinaldi, director of the Developmental Disabilities Training Project, called an emergency meeting of the project staff.
"You've all been working really hard on the training manual," he said. "I hate to put this on you right now, but I've got to tell you that I just had word. The feds are sending in their evaluator. He's going to be here next week, and we've got to be ready."
Amid the groans, Jane Carlin, a staff trainer, spoke up. "What's the big problem?" she asked. "We've been providing a training session every week. We've had workshops on developmental disabilities for the teachers, for citizens; we've had the TV show, now we've got a manual for parents. It seems to me we're in great shape. So what's the problem?"
"Well, I was kind of putting this part off," Rinaldi answered. "They sent along the new evaluation form so we can complete the self-study before the evaluator gets here. That's the tough part. We're really going to have to dig to get the information ready."
"What kind of information do we need? Remember, we did that pretest and posttest with all the people at the workshops. We've got a lot of data on the learning effects from the workshop. Of course, it's not that easy to do with the TV program."
"That's the least of our problems, Jane. Remember this is a training project. What they want is information on all participants in any training workshop. They want the ages of the trainees, their sex, their employment, their income--all the demographic stuff. I just didn't think about all that because the form we used last year didn't ask for it. You see, we're using the same kind of evaluation form as continuing ed programs are. It doesn't make any sense, but I'll bet we can dig up that information somehow."
"Wait a minute!" George Steinberg called out. "I worked my tail off on that TV program, and there's no way in the world I'll ever be able to even guess who watched it. Does that mean the whole thing didn't happen? Does that mean I get a grade of zero? I flunk?"
"Now, George, you know it isn't like that. They just like to have the information so they can put it together with the data from all the other projects they funded. They've got to show results, just as we do."
"That's fine to say, John, but if they're going to evaluate us on how many men, women, and children show up at our sessions, we should do the kind of stuff that can lend itself to what they're looking for. I'm feeling as if my work just isn't going to make the grade."
"I'm starting to feel that way, too" Carlin said. "What's the point of doing one thing if they're evaluating something else?"
"Wait a minute, everybody," Rinaldi responded in frustration. "We're getting way off the track here. These people aren't here to tell us what we should and should not do. They're not going to "grade" us on what we did. One of the things they ask about in an open-ended question is the content of the program, the kinds of interventions we did. There's no problem there. The problem is just in putting together the data they want, the demographic characteristics and all that. Now, I've got most of it somewhere. All I need is for somebody to volunteer to help me dig through the files and see what we can find that might relate to some of the questions they're asking. We'll be able to get the sex of participants by looking at their names on those address cards we had them fill out. The ages will be hard. We'll have to guess at that."
Write a case analysis addressing the following questions. Use the list of questions as a guideline, answering them one at a time for clarity. Please be thorough when answering the questions. There is no set length for this paper but again, give informed, thoughtful answers.
1. What concepts from your readings and the online lectures are illustrated by this case?
2. From the case, what are the outcome objectives and the process objectives for this program?
3. What data should the staff have been collecting for this program related to both outcome and process objectives?
4. How might the agency have avoided this "emergency" situation?
5. How might the funding source have helped avoid the situation?
6. How should the staff deal with this immediate situation? For instance, how should they go about estimating the size and composition of the TV audience?
7. Why does the funding agency need demographic information?
8. Will their demographic estimates be valid, if every agency they funded faces the same "emergency"? Why or why not?
9. What impact might invalid information have on the funding source, especially if every agency faces the same "emergency"?
10. What lessons does this case study teach you about information management that you will apply to your own agency settings?