Case Scenario: Did Your Mother's Breakfast Determine Your Sex?
Read the original source on the companion website, https:// www.cengage.com/statistics/Utts4e.
You've probably heard that "you are what you eat," but did it ever occur to you that you might be who you are because of what your mother ate? A study published in 2008 by the British Royal Society seemed to find just that. The researchers reported that mothers who ate breakfast cereal prior to conception were more likely to have boys than mothers who did not (Mathews et al., 2008). But 9 months later, just enough time for the potential increased cereal sales to have produced a plethora of little baby boys, another study was published that dashed cold milk on the original claim (Young et al., 2009). The dispute was based on something statisticians call multiple testing, which can lead to erroneous findings of statistical significance.
The authors of the original study had asked 740 women about 133 different foods they might have eaten just before getting pregnant. They found that 59% of the women who consumed breakfast cereal daily gave birth to a boy, compared to only 43% of the women who rarely or never ate cereal (https://www.cbsnews.com/stories/ 2008/04/22/health/webmd/main4036102.shtml). The result was highly statistically significant, but almost none of the other foods tested showed a statistically significant difference in the ratio of male to female births. As previously discussed, statistical significance is how statisticians assess whether a difference found in a sample, in this case of 740 women, is large enough to conclude that the difference is likely to represent more than just chance. But sometimes what looks like a statistically significant difference is actually a false positive-a difference that looks like it wasn't due to chance when it really was.
The more differences that are tested, the more likely it is that one of them will be a false positive. The criticism by Young et al. was based on this idea. When 133 food items that in fact do not affect the sex of a baby are all tested, it is likely that at least one of them will show up as a false positive, showing a big enough difference in the proportion of male to female births to be statistically significant when in fact the difference is due to chance. The authors of the original study defended their work (Mathews et al., 2009). They noted that they only tested the individual food items after an initial test based on total pre-conception calorie consumption showed a difference in male and female births. They found that 56% of the mothers in the top third of calorie consumption had boys, compared with only 45% of the mothers in the bottom third of calorie consumption. That was one of only two initial tests they did; the other had to do with vitamin intake. With only two tests, it is unlikely that either of them would be a false positive. Unfortunately the media found the cereal connection to be the most interesting result in the study, and that's what received overwhelming publicity. The best way to resolve the debate, as in most areas of science, is to ask the same questions in a new study and see if the results are consistent. The authors of the original study have stated their intention to do that.
Moral of the Story: When you read about a study that found a relationship or difference, try to find out how many different things were tested. The more tests that are done, the more likely it is that a statistically significant difference is a false positive that can be explained by chance. You should be especially wary if dozens of things are tested and only one or two of them are statistically significant.
Definitions: Multiple testing or multiple comparisons in statistics refers to the fact that researchers often test many different hypotheses in the same study. This practice may result in statistically significant findings by mistake, called false positive results. Sometimes this practice is called data snooping because researchers snoop around in their data until they find something interesting to report.