Valid and Reliable Tools
The goal of an evaluation is to determine the success of an intervention, a new process, the launch of a new technology, patient satisfaction, or any number of things. Surveys are a popular tool for gathering this type of information. For the results of the evaluation to be meaningful, however, the survey used must be both reliable and valid. What does that entail? A reliable instrument is one that would yield similar results when given to different groups under identical circumstances.
For example, if a survey was given to nurses on the use of a certain piece of technology, all respondents would understand the phrasing of the questions the same way. Validity refers to how well the instrument actually measures what it is intended to measure. Determining the reliability and validity of a survey instrument can be complicated and involves the use of statistics. For this reason, many researchers opt to use instruments that are already developed and tested.
For this Discussion, you consider survey instruments that would be appropriate to use in specific situations.
The following scenarios will be used for this week's Discussion:
Scenario 1: A large hospital intends to implement a computerized provider order entry (CPOE) system. In order to get a good idea of its effects, the hospital's leadership has asked for an evaluation of the CPOE system's impact 90 days after its initial implementation.
Scenario 2: Years ago, the primary hospital for a large, rural county distributed personal data assistants (PDAs) to all of its physicians in an attempt to modernize. After looking at many other more up-to-date mobile systems, physicians and hospital leaders are curious about how their current PDA-based system performs.
Scenario 3: The informatics department of one of North America's largest hospitals is conducting an internal review of its health information technology systems. This review will evaluate the need for any changes to its systems and may serve as justification for different budgetary allocations. Because of its sheer size and the number of personnel it affects, the hospital's electronic health record system will be a pivotal point of the review.
To prepare:
Review this week's Learning Resources on reliability and validity.
Review the AHRQ Evaluation Survey Compendium.
Review the scenarios presented above.
Using the "Locate a Survey for your Project" tool available on the AHRQ website, identify a survey tool that would be appropriate for use for each scenario.
Reflect on the specific characteristics of a valid, reliable survey tool.
a minimum of 550 words in APA format with 3 references that include the unique survey tool youidentified for each scenario and a justification for your selections
Required Readings
Friedman, C. P., & Wyatt, J. C. (2010). Evaluation methods in biomedical informatics (2nd ed.). New York, NY: Springer Science+Business Media, Inc.
Chapter 5, "Measurement Fundamentals" (pp. 113-144)
Anderson, E. F. (2011). A case for measuring governance. Nursing Administration Quarterly, 35(3), 197-203.
Retrieved from the Walden Library databases.
Kaphingst, K. A., Kreuter, M. W., Casey, C., Leme, L., Thompson, T., Cheng, M. R., et al. (2012). Health literacy INDEX: Development, reliability, and validity of a new tool for evaluating the health literacy demands of health information materials. Journal of Health Communication, 17(Supp 3), 203-221.
Penfold, R. B., Kullgren, J. T., Miroshnik, I., Galbraith, A. A., Hinrichsen, V. L., & Lieu, T. A. (2011). Reliability of a patient survey assessing cost-related changes in health care use among high deductible health plan enrollees. BMC Health Services Research, 11(1), 133-143.
Seto, I., Foisy, M., Arkison, B., Klassen, T., & Williams, K. (2012). The evaluation of an evidence-based clinical answer format for pediatricians. BMC Pediatrics, 12, 34-41.
Agency for Healthcare Research and Quality. (n.d.). Health IT survey compendium. Retrieved from
https://healthit.ahrq.gov/portal/server.pt/community/health_it_tools_and_resources/919/health_it_survey_compendium/27874
Required Media
Laureate Education (Producer). (n.d.f). Reliability and validity. Retrieved from CDN database. (NURS 6431)
This video explores the concepts of reliability and validity in measurement tools. The video stresses the importance of reliability and validity when selecting an appropriate tool to evaluate a PICO question.