Evaluating Significance of Findings
Scenario 1
A results report that does not find any effect and also has small sample size (possibly no effect detected due to lack of power).
A one-way analysis of variance was used to test whether a relationship exists between educational attainment and race. The dependent variable of education was measured as number of years of education completed. The race factor had three attributes of European American (n = 36), African American (n = 23) and Hispanic (n = 18). Descriptive statistics indicate that on average, European Americans have higher levels of education (M = 16.4, SD = 4.6), with African Americans slightly trailing (M = 15.5, SD = 6.8) and Hispanics having on average lower levels of educational attainment (M = 13.3, SD = 6.1). The ANOVA was not significant F (2,74) = 1.789, p = .175, indicating there are no differences in educational attainment across these three races in the population. The results of this study are significant because they shed light on the current social conversation about inequality.
Scenario 2
Statistical significance is found in a study, but the effect in reality is very small (i.e., there was a very minor difference in attitude between men and women). Were the results meaningful?
An independent samples t test was conducted to determine whether differences exist between men and women on cultural competency scores. The samples consisted of 663 women and 650 men taken from a convenience sample of public, private, and non-profit organizations. Each participant was administered an instrument that measured his or her current levels of cultural competency. The cultural competency score ranges from 0 to 10, with higher scores indicating higher levels of cultural competency. The descriptive statistics indicate women have higher levels of cultural competency (M = 9.2, SD = 3.2) than men (M = 8.9, SD = 2.1). The results were significant t (1311) = 2.0, p <.05, indicating that women are more culturally competent than are men. These results tell us that gender-specific interventions targeted toward men may assist in bolstering cultural competency.
Critically evaluate the two scenarios above based upon the following points:
1. Critically evaluate the sample size.
2. Critically evaluate the statements for meaningfulness.
3. Critically evaluate the statements for statistical significance.
4. Based on your evaluation, provide an explanation of the implications for social change.