Question: A Winning Confidence Interval Loses in Court Gastwirth (1988, p. 495) describes a court case in which Sears, Roebuck and Company, a large department store chain, tried to use a confidence interval to determine the amount by which it had overpaid city taxes at stores in Inglewood, California. Unfortunately, the judge did not think the confidence interval was appropriate and required Sears to examine all the sales records for the period in question. This case study provides an example of a situation where the answer became known, so we can compare the results from the sample with the true answer. The problem arose because Sears had erroneously collected and paid city sales taxes for sales made to individuals outside the city limits. The company discovered the mistake during a routine audit, and asked the city for a refund of $27,000, the amount by which it estimated it had overpaid. Realizing that it needed data to substantiate this amount, Sears decided to take a random sample of sales slips for the period in question and then, on the basis of the sample proportion, try to estimate the proportion of all sales that had been made to people outside of city limits. It used a multistage sampling plan, in which the 33-month period was divided into eleven 3-month periods to ensure that seasonal effects were considered. It then took a random sample of 3 days in each period, for a total of 33 days, and examined all sales slips for those days. Based on the data, Sears derived a 95% confidence interval for the true proportion of all sales that were made to out-of-city customers.
The confidence interval was .367 .03, or .337 to .397. To determine the amount of tax Sears believed it was owed, the percentage of out-of-city sales was multiplied by the total tax paid, which was $76,975. The result was $28,250, with a 95% confidence interval extending from $25,940 to $30,559. The judge did not accept the use of sampling despite testimony from accounting experts who noted that it was common practice in auditing. The judge required Sears to examine all of the sales records. In doing so, Sears discovered that about one month's worth of slips were missing; however, based on the available slips, it had overpaid $26,750.22. This figure is slightly under the true amount due to the missing month, but you can see that the sampling method Sears had used provided a fairly accurate estimate of the amount it was owed. If we assume that the dollar amount from the missing month was similar to those for the months counted, we find that the total Sears was owed would have been about $27,586. Sampling methods and confidence intervals are routinely used for financial audits. These techniques have two main advantages over studying all of the records. First, they are much cheaper. It took Sears about 300 person-hours to conduct the sample and 3384 hours to do the full audit. Second, a sample can be done more carefully than a complete audit. In the case of Sears, it could have two well-trained people conduct the sample in less than a month. The full audit would require either having those same two people work for 10 months or training 10 times as many people. As Gastwirth (1988, p. 496) concludes in his discussion of the Sears case, "A well designed sampling audit may yield a more accurate estimate than a less carefully carried out complete audit or census." In fairness, the judge in this case was simply following the law; the sales tax return required a sale-by-sale computation.