DQ 1
What are the most effective strategies for managing quality control on quantitative methods in program evaluation? Why are they the most effective?
When an evaluator is conducting a quantitative evaluation on an organization it is extremely important that effective strategies are applied to manage quality control. Key stakeholders are relying on the evaluator to provide reliable and valid information based on the data that has been gathered. As reported by Min, Ko, Cho, Jeong, Lee,Chun,and Cho (2015), systematic errors occur frequently in clinical laboratory test results and should be detected and corrected. This helps to ensure the validity and reliability of the results.
Min et. al (2015) goes on to report that, in conclusion, the newly developed quantitative quality control procedure (QQCP) can analyze systematic errors quantitatively and be applied to every run. As with qualitative evaluations triangulation is another quality control method that can be applied to compare the results of the data as a way of maintaining the validity and reliability.
As reported by Tonkin-Cline, Anthierens, Hood, Yardley, Cals, Francis, Coenen, van der Velden, Godycki-Cwirko, Llor, Butler, Verheij, Goossens, Little (2016), a triangulation protocol can also enhance the validity of findings and assess whether data agree (convergence), complement one another (complementarity) or contradict each other (dissonance). Once these things have been determined it makes it easier to make the necessary changes.
Min, W., Ko, D., Cho, E. J., Jeong, T., Lee, W., Chun, S., & Cho, H. (2015). A novel quantitative evaluation method for quality control results. Clinica Chimica Acta, 451(Part B), 175-179. doi:10.1016/j.cca.2015.09.026
Tonkin-Crine, S., Anthierens, S., Hood, K., Yardley, L., Cals, J. L., Francis, N. A., & ... Little, P. (2016). Discrepancies between qualitative and quantitative evaluation of randomised controlled trial results: achieving clarity through mixed methods triangulation. Implementation Science, 111-8. doi:10.1186/s13012-016-0436-0
DQ 2
Explain why quantitative methods would prove advantageous over qualitative methods in program evaluation. Are there drawbacks to using quantitative methods? If so, which ones are of greatest concern? Why?
Quantitative evaluation methods can be less expensive than qualitative methods and allow evaluators to provide results quickly. The cost savings can be made in reduced man-hours as interviews or observations are not required replaced by surveys and questionnaires (Rao & Woolcock, 2003).
The instruments used in quantitative evaluation do not require coding (Choy, 2014). Quantitative instruments provide numeric or categorical data that can be analyzed after data cleaning but coding is not necessary. Again, this saves time. Surveys and questionnaires can be distributed and returned quickly to more participants. Qualitative methods are generally more time and labor intensive as researchers interview, observe and review documents triangulating data (Royse, Thyer, & Padgett, 2016).
Although cost effective quantitative data is not always appropriate. The findings provide statistical results. This may be exactly what is needed if stakeholders want a picture of statistical outcomes, such as the number of program users successfully employed. The evaluation technique is summative.
However, if this outcome is already known and the stakeholders want to know what user experiences were for those who succeeded the evaluation must be qualitative, a formative focus. The greatest consideration when selecting an evaluation method is the research question posed by the stakeholders.
References
Choy, L. T. (2014). The Strengths and Weaknesses of Research Methodology: Comparison and Complimentary between Qualitative and Quantitative Approaches. IOSR Journal Of Humanities And Social Science, 19(4), 99-104.
Rao, V., & Woolcock, M. (2003). Integrating Qualitative and Quantitative Approaches in Program Evaluation. In F. Bourguignon, & L. A. Silva, The impact of economic policies on poverty and income distribution: Evaluation techniques and tools (pp. 165-190). New York , NY: Oxford University Press.
Royse, D., Thyer, B. A., & Padgett, D. K. (2016). Program evaluation: An introduction (6th ed.). Boston, MA: Cengage Learning.