Question: The Infamous Literary Digest Poll of 1936
Before the presidential election of 1936, a contest between Democratic incumbent Franklin Delano Roosevelt and Republican Alf Landon, the magazine Literary Digest had been extremely successful in predicting the results in U.S. presidential elections. But 1936 turned out to be the year of its downfall, when it predicted a 3-to-2 victory for Landon. To add insult to injury, young pollster George Gallup, who had just founded the American Institute of Public Opinion in 1935, not only correctly predicted Roosevelt as the winner of the election, but also predicted that the Literary Digest would get it wrong. He did this before they had even conducted their poll! And Gallup surveyed only 50,000 people, while the Literary Digest sent questionnaires to 10 million people (Freedman et al., 1991, p. 307).
The Literary Digest made two classic mistakes. First, the lists of people to whom they mailed the 10 million questionnaires were taken from magazine subscribers, car owners, telephone directories, and, in a few instances, lists of registered voters. In 1936, those who owned telephones or cars or subscribed to magazines were more likely to be wealthy individuals who were not happy with the Democratic incumbent.
Despite what many accounts of this famous story conclude, the bias produced by the more affluent list was not likely to have been as severe as the second problem (Bryson, 1976). The main problem was nonresponse bias caused by volunteer response. The Literary Digest received 2.3 million responses, a response rate of only 23%. Those who felt strongly about the outcome of the election were most likely to respond. That included a majority of those who wanted a change: the Landon supporters. Those who were happy with the incumbent were less likely to bother to respond.
Gallup, on the other hand, knew the value of random sampling. He not only was able to predict the election, but also predicted what the results of the Literary Digest poll would be to within 1%. How was he able to predict the results of the Literary Digest survey? According to Freedman et al. (1991, p. 308), "he just chose 3,000 people at random from the same lists the Digest was going to use, and mailed them all a postcard asking them how they planned to vote."