Question 1) What were the alternative methods used by the FTC and the merging firms to determine whether or not an Office Depot and a Staples outlet were in the same geographical market?
Question 2) How did this differentiation in geographical market measurement affect the pricing behavior estimated by the FTC and by the merging firms? What is the economic term we commonly use for "price sensitive" as used in the paper?
Question 3) Which of the two studies do you find more compelling? Would you allow the merger? Has your company been involved in a similar court injunction (if no, choose a similar example outside your firm)? Identify the case and how it is similar.
Econometric Analysis in FTC versus STAPLES
Prepared Remarks of
JONATHAN B. BAKER(1)
Director, Bureau of Economics
FEDERAL TRADE COMMISSION
Before
American Bar Association's Antitrust Section
Economics Committee
Willard Hotel
July 18, l997
Revised: March 31, 1998
At the end of June, a federal district court in Washington, D.C. granted the Federal Trade Commission's (FTC's) request for a preliminary injunction blocking the Staples-Office Depot merger.(2) The proposed transaction would have combined Staples and Office Depot, two of the three leading office superstore chains. The FTC presented extensive documentary evidence from the merging firms' files at the hearing. These documents demonstrated that the two superstore chains charge lower prices for consumable office supplies in cities where they directly compete relative to prices in cities where the merging firms do not face each other head-to-head. The documents also showed that superstore competition is the main reason for this pricing policy. For example, the merging superstore chains each moved stores into "price zones" with lower prices in response to entry by rival superstores, but not in response to new competition by other retailers.(3) Indeed, Office Depot placed those locations free from competition from another superstore in a price zone termed "non-competitive" (without regard to whether other retailers nearby sold office supplies).(4) The court relied heavily on this documentary evidence in explaining its decision.
Econometric evidence was also an important part of the case for both sides in the litigation. The FTC confirmed what the documents showed through a systematic empirical study of Staples' pricing presented in court by Professor Orley Ashenfelter, our econometric expert. The FTC also presented an econometric study of the rate that Staples historically passed through cost savings to consumers in the form of lower prices. For their part, the merging firms offered alternative statistical analyses of pricing, as well as econometric studies of the determinants of Staples price-cost margins and the effect on revenues at Staples stores of nearby store openings by possible rivals. My remarks today describe the motivation and methods behind the FTC's econometric analyses.
I. The importance of the Staples case:
The Staples litigation was important because it put into play the four main initiatives in merger analysis undertaken by the federal antitrust enforcement agencies over the past decade. First, to explain how the merger would harm competition, the Commission looked to the unilateral competitive effects theory for mergers among sellers of differentiated products, set forth in the Merger Guidelines.(5) Second, the litigants proffered extensive econometric analyses, primarily going to the importance of localized competition between the merging firms and the constraint they place upon each other.(6) Third, the extensive courtroom discussion of the significance of efficiencies alleged by the merging firms was conducted against the background of newly-released revisions to the Merger Guidelines, which set forth a new analytical approach to answering that question.(7) Finally, the Commission and the merging firms contested whether new competition, particularly product line extension by non-office superstores selling office supplies, would solve the competitive problem from merger -- thus implicating the "entry likelihood" analysis in the Merger Guidelines, which some courts have misunderstood.(8)
These four government initiatives emerged unscathed in Judge Thomas F. Hogan's opinion. Although they were largely not treated explicitly in the written decision, the opinion that one might say hides behind the words Judge Hogan wrote bolsters each.
First, although the court did not refer to the unilateral effects theory by name, Judge Hogan employed its logic in explaining why he found an office superstore submarket and why the merger would have harmed competition.(9) In defining the product market, the opinion recognized that office superstore chains provide the primary competitive constraint on each other's pricing. "While it is clear to the Court that Staples and Office Depot do not ignore sellers such as warehouse clubs, Best Buy, or Wal-Mart, the evidence clearly shows that Staples and Office Depot each consider the other superstores as the primary competition."(10) And in explaining why the merger would lead to adverse competitive effects, the court adopted the reasoning of the localized competition theory for mergers among sellers of differentiated products set forth in the Merger Guidelines. Judge Hogan observed that "direct evidence shows that by eliminating Staples' most significant, and in many markets only, rival, this merger would allow Staples to increase prices or otherwise maintain prices at an anti-competitive level."(11) Thus, when the written opinion appeals to the "practical indicia" for defining submarkets listed by the Supreme Court in Brown Shoe,(12) the hidden opinion treats this approach as a legal hook for reaching unilateral competitive effects from a merger among the sellers of close substitutes.(13) Judge Hogan did not return to the past by defining a narrow market; he instead used the old construct of a submarket to help articulate a contemporary perspective.
Second, Judge Hogan's hidden opinion supports the government's use of econometric evidence, though the court did not trumpet doing so. The opinion never uses the term, presumably in a conscious effort to downplay novelty in order to avoid creating an issue for appeal. Yet Judge Hogan demonstrably relied on econometric evidence in one instance,(14) when he stated that "in this case the defendants have projected a pass through rate of two-thirds of the savings while the evidence shows that, historically, Staples has passed through only 15-17%."(15) The sole basis in the record for the 15-17% figure is the testimony of the FTC's econometric expert as to the conclusions of his statistical analysis of the pass-through rate.
Third, Judge Hogan approached efficiencies in a diffident way, by first pointing out that if old Supreme Court precedents control, the efficiency defense may not be viable.(16) But the opinion hidden behind this unassuming approach supports the government's methodology for reviewing claimed efficiencies. After nodding to the old Supreme Court cases, Judge Hogan examines efficiencies with an approach that tracks the recent Merger Guidelines revisions. The court refused to accept alleged cost savings when "the defendants did not accurately calculate which projected cost savings were merger specific and which were, in fact, not related to the merger."(17) Judge Hogan dismissed much of the defendants' projected cost savings on the ground that they are "in large part unverified, or at least the defendants failed to produce the necessary documentation for verification."(18) And in finding "that the defendants' projected pass through rate -- the amount of the projected savings that the combined company expects to pass on to consumers in the form of lower prices -- is unrealistic,"(19) the court followed the Merger Guidelines in focusing on whether consumers would obtain the benefit of the efficiencies.
Finally, in supporting its conclusion that entry would not solve the competitive problem, the written opinion emphasized the factual basis for that finding and the weaknesses in the defendants' evidence. Yet, in a matter-of-fact way, the court adopted the perspective of the Merger Guidelines. Judge Hogan recognized as the legal standard whether entry "would likely avert anticompetitive effects" from the acquisition by acting a constraint on the merged firms' prices.(20) Here the court accepted that entry matters under Clayton Act §7 insofar as it would solve the competitive problem from the merger. Unlike some other courts, Judge Hogan did not see his task as assessing the height of barriers to entry in the abstract, unrelated to the transaction before him.(21) Rather, Judge Hogan properly compared how the office superstore market would likely look after the proposed transaction (including the competitive significance of any additional entry that the merger would call forth) with the likely evolution of the market absent the proposed acquisition. This perspective on entry was reinforced by similar comparisons of the market with the merger to the but-for world in Judge Hogan's analysis of efficiencies -- by refusing to accept efficiency claims that were not merger-specific -- and in his analysis of competitive effects. In the latter context, the court pointed out that when the opinion discusses "raising" prices it makes that comparison "with respect to where prices would have been absent the merger," regardless of whether the prices represent "an increase from present price levels."(22)
The remainder of my remarks will highlight the second of the four government initiatives by describing the use of econometrics by the FTC in the Staples litigation.(23) I will first discuss the FTC's econometric studies that examined the extent of localized competition between the merging firms. I will then turn to the Commission's econometric analysis of the extent to which the merged firm would pass on cost savings from the acquisition to buyers. I will conclude by drawing out some tentative lessons for antitrust policy, and for the process of relying on econometric evidences in merger investigations.
II. Pricing Studies:
Most of the econometric effort in the investigation and litigation focused on studies of pricing. Indeed, the pricing documents of the merging firms are what first attracted the FTC staff's attention. We saw, and later introduced into court, documents that demonstrated that Staples and Office Depot each set prices and created price zones primarily based on competition from other office superstore chains (its merger partner and OfficeMax). The documents showed that Staples expected that the merger would ease competitive pressure from Office Depot, allowing Staples to increase margins by an amount that the FTC's primary economic expert, Dr. Frederick Warren-Boulton, later translated into an average 5-10% price increase on office supplies in overlap markets.
The non-econometric evidence further demonstrated that Staples prices were significantly lower in cities where Staples competed with Office Depot than in what Staples termed "non-competitive" price zones, where Staples faced no other superstore chains. Similarly, Staples prices were lower in three-superstore chain cities than in cities where Staples and OfficeMax both had a presence but Office Depot did not. As the FTC's economic expert later testified, this pricing data suggested that the merger, by removing Office Depot from the market, would raise price on average by more than 9% in overlap markets.(24)
Our initial econometric estimates, made during the investigation, were aimed at confirming systematically what we thought we had learned from the party documents: that Staples prices were lower when Office Depot had a greater presence nearby. We had weekly data from the parties, covering more than 400 Staples stores (spread over more than 40 cities) for over eighteen months. The data included prices for a number of individual office supply products (defined by stock keeping units (SKU's)) as well as a price index for consumable office supplies created by the merging firms' economic expert. We conducted most of our analyses on monthly aggregates, in part because we initially were unable to sample on a weekly basis some variables we wished to add to the model.(25)
The main idea of the econometric analysis of pricing was to look at how Staples' prices varied from one store to the next or over time as the number of nearby Office Depot stores varied.(26) We used the results of that analysis to simulate the effect of the merger in two alternative ways. One procedure was proposed by the defendants' economic expert: treating the merger as closing down Office Depot stores near to Staples stores. The alternative approach took the view that the merger would convert Office Depot stores into Staples stores.
A. Cross-section vs. fixed effects estimates
Our initial internal analyses pooled what could be learned by comparing prices across the stores in our sample with what could be learned by comparing price changes over time as more superstores enter a market.(27) In the data, pricing across markets varied more than pricing over time, so the estimates using pooled data were dominated by comparisons across markets. Accordingly, we were essentially employing a "cross-sectional" statistical approach that adopted the perspective of the merging firms' documents: we determined the effects of competition between Staples and Office Depot by comparing the price Staples charged at stores facing competition from nearby Office Depot stores with the price Staples charged at stores free from Office Depot rivalry.
The FTC staff's internal cross-sectional estimates were similar to the cross-sectional estimates that the FTC's econometric expert later reported: prices were substantially lower where Staples competed with Office Depot, and a merger between the two would likely allow Staples to raise the average price of consumable office supplies by more than 7%.(28) Moreover, the average simulated price increase for a price index limited to what Staples termed "price sensitive items" (such as copy paper, popular brands of pens, and 1/3 cut file folders) was more than double the predicted increase for the consumable office supply price index as a whole.(29)
The main criticism of this approach offered by the merging firms and their economic expert was that the cross-sectional comparison was biased toward finding a greater price effect of head-to-head competition than actually existed.(30) They insisted that Staples prices were high in single superstore markets and other markets where Office Depot did not compete because, on average, costs other than those we could measure and control for in our equations -- perhaps resulting from local zoning provisions or congestion -- were high in those markets. They asserted that these higher costs simultaneously led Staples to raise price above what it charged elsewhere and discouraged Office Depot from entering.
On the surface, this argument seemed plausible. It is a common criticism of cross-sectional studies to question whether the results are biased because the econometrician is unable to observe and control for important differences across markets and those differences are correlated with the variables whose effect is at issue.(31) And if prices were at cost in all markets -- as the merging firms contended -- then the only way we could observe higher prices in markets with less superstore competition, as we did, is if the costs we were unable to control for were higher in those markets. Unfortunately for the parties, this theoretical possibility had negligible support in their documents. Our extensive review turned up no evidence of important unobservable cost variables affecting pricing, except in one city. Based on these documents, which did not support the merging firms' claims, we believed that omitted variables did not bias our cross-section econometric analyses.
The merging firms' economic expert sought to test the omitted variable bias hypothesis statistically, notwithstanding the absence of support for that theory in the pricing documents. He proposed to compare the cross-section estimates with those derived from a "fixed effects" model. The fixed effects model incorporates "indicator" (or "dummy") variables for the individual stores. It controls for the possibility of omitted variable bias because the unobservable costs -- whose variation across regions were allegedly affecting both Staples pricing and rival entry decisions -- were likely not to vary over time at any one location. That is, if roads were congested or zoning approvals difficult to obtain in some area at the beginning of the sample, these local features were likely to continue to be observed eighteen months later. Under such circumstances, the effect of rivalry from Office Depot on Staples pricing can be isolated by looking at what happened to Staples prices in locations where Staples stores were free or largely free from such competition at the beginning of the sample but faced more nearby Office Depot stores at the end of the sample. By including store fixed effects, comparisons of prices across stores are effectively removed from the sample; the estimated effect of Office Depot rivalry on Staples pricing comes solely from pricing variation within markets over time. Accordingly, if the fixed effects model gives similar estimates to the cross-section model, then the relationship observed in the cross-market comparisons is unlikely to have been biased by the failure to control for unobservable cost variation across stores.
This test could not be definitive, however, because the difference between the cross-section and fixed effects estimates may not measure cleanly the magnitude of the omitted variable bias in the cross-section regression. Fixed effects models tend to exaggerate "errors-in-variables" bias -- the difficulty in detecting statistically the influence of an explanatory variable when that variable is measured with error.(32) The measurement error at issue could be technical (e.g. recording the wrong opening date for Office Depot stores) or conceptual (e.g. weighting nearby and distant Office Depot locations improperly in computing a variable intended to reflect the intensity of rivalry with Office Depot).(33) Thus, if rivalry from Office Depot appeared to have less influence on Staples pricing in the fixed effects regression than in the cross-section regression, there were two possible explanations: that the cross section results were biased upward because cost variables correlated with Staples pricing and Office Depot entry were omitted from the cross-section equation, or that the fixed effects results were biased downward because the variables controlling for the extent of rivalry from Office Depot measured that rivalry with error.(34) Under the first explanation, the test of the omitted variable bias hypothesis is accurate, revealing a problem with the cross-section estimates. Under the second explanation, the problem is with the test not with the cross-section result.
Well before the Commission decided to challenge this transaction, the defendants' economic expert reported that the simulated effect of the merger was just under 1% when the effect of rivalry from Office Depot on Staples prices was measured with a fixed effects model rather than a cross-section model. This fixed effects estimate was nearly an order of magnitude less than the cross-section estimates obtained by the Commission staff. At this point, the interpretation of the pricing data appeared to shape up as an argument about whether to prefer cross-section or fixed effects models for estimating the price effect of rivalry between the merging firms. The cross-section results were consistent with the documents, while the fixed effects results may have controlled for omitted variables that might bias the cross-section analyses at the price of exacerbating an errors-in-variables bias. Given the powerful evidence in the merging firms' documents about the price-depressing effect of rivalry between the two -- the same evidence later highlighted by Judge Hogan -- and the absence of any indication in the documents of important omitted variables influencing Staples pricing and Office Depot entry, we interpreted the lower fixed effects estimates as reflecting measurement error -- as a flawed test of the omitted variable bias hypothesis -- rather than as disproving the cross section estimates.
Through further data analysis, our interpretation was shown to be correct. At the trial Professor Orley Ashenfelter, the FTC's econometric expert, described simulations of the impact of the merger on Staples prices based on his fixed effects regressions that were similar to those based on cross-section analyses.(35) He highlighted two main problems with the fixed effects study presented by the merging firms' expert; correcting these problems moved the simulated average nationwide price increase from just under 1% to the range of 7-9% (depending on how the transaction was modeled in the simulation).(36)
The first problem was a type of conceptual measurement error.(37) The merging firms' expert had measured the presence of Office Depot (and, similarly, the presence of all other actual or potential rivals to Staples) in three non-overlapping concentric circles: one 0 to 5 miles from a Staples store, one 5 to 10 miles away, and one 10 to 20 miles away. This was not on its face an implausible approach for capturing the competitive significance of rivalry from Office Depot, but it did not share the perspective of the documentary evidence that the merging firms established price zones commonly encompassing entire metropolitan areas within which prices were nearly uniform.(38) Our econometric expert showed that it was statistically important to do what the price zone doucments suggested: include in addition a variable based on the number of Office Depot stores within the metropolitan statistical area (MSA).(39) Adding the MSA-based variable along with the concentric circle variables had the effect of tripling or quadrupling the simulated price effect of the merger -- moving the simulated price increase from just under 1% to a range between 2.5% and 3.7% (varying with certain technical differences in the method of simulation).(40)
The other problem with the fixed effects study presented by the merging firms' expert was a sample selection bias. This bias resulted from the arbitrary exclusion of observations in California, Pennsylvania and certain other areas.(41) When the excluded stores were included, the simulated price effect of the merger nationwide more than doubled from the 2.5% to 3.7% range, now reaching a range between 6.5% and 8.6%.
The main response of the defendants' economic expert was to argue that the FTC's expert had inappropriately included data for California stores in the same regression model with the data for the rest of the U.S., given that a statistical test (Chow test) showed that the two subsamples behaved differently.(42) The FTC's econometric expert agreed in principle, but demonstrated that this criticism did not overturn his conclusion that simulations of the merger using the fixed effect regression model suggest that prices would rise on average more than 6% in overlap markets. Indeed, when he adjusted his methodology to address this concern, he actually found to higher simulated price increases than before. The adjustment involved estimating the regression model separately for the two relevant groups of stores (the California locations in the subsample identified by the defendants' expert, and the rest of the U.S.), simulating the price effect separately for each subsample, and computing a nationwide average as a weighted average of the two regional estimates. For example, the FTC's expert originally reported simulations using one regression model that generated a 7.6% average price increase. Using the alternative methodology that responded to this criticism, the same model implied a nationwide average price rise of 9.8%, over two percentage points higher.(43)
B. Other econometric issues
Three other econometric issues involving the pricing studies were raised but not fully addressed by both sides under the time pressures of litigation. The first was the reliability of simulations out of sample, an issue with the simulations both sides conducted based on fixed effects models.(44) Over the less than two years in the data, any given Staples store might see the entry of a small number of Office Depot stores nearby, but rarely as many as five. Yet Office Depot has more than five locations in many metropolitan areas (including substantially more than five locations in Los Angeles). In order to simulate the merger in MSAs with large numbers of Office Depots using a fixed effects model, it is necessary to extrapolate the regression model out of sample, adding to the uncertainty of the predictions.(45)
The second issue was the potential endogeneity of entry. The regression model treats the addition of a store by Staples or any of its potential rivals as an exogenous event, unrelated to the price Staples charges. Yet it is possible that a high Staples price encourages expansion and entry by Staples and perhaps other firms as well. The defendants' economic expert raised this possibility as one reason the FTC expert's results might be biased, but did not press the point.(46) For our part, the FTC staff had preliminary results, not part of the trial record, correcting for this problem by using instrumental variables to estimate the regression model. This correction led to simulated price increases roughly double those based on regressions estimated using ordinary least squares.(47)
The third issue was whether fixed effects are the best way to account for the possibility of omitted store-specific cost variables correlated with both Staples prices and Office Depot entry. Both sides relied on models that used store fixed effects for this purpose. This modeling strategy assumes that any such omitted variables do not vary over time. On the eve of trial, the defendants' economic expert proposed instead accounting for omitted variables in a new way, with store-specific time trends along with store fixed effects. This is a more stringent test of whether the cross-section results are biased as a result of omitted cost variables than the fixed effects model, but it places an even greater premium on measuring the independent variables properly.(48)
II. Pass-Through of Cost Changes
The FTC's econometric expert also testified to a statistical analysis of the rate at which Staples historically passed on firm-specific cost reductions to consumers. The merging firms' expert had framed the issue by asserting that Staples typically reduced price by two-thirds of any cost reduction, though he did not present a data analysis in support of this conclusion. In response, the FTC's economic expert, Dr. Frederick Warren-Boulton, pointed out the importance of distinguishing between firm-specific and industry-wide cost shocks. He argued that pass through rate for industry-wide cost savings was likely greater than the rate for firm-specific savings; in the former case competition would force prices down. Yet the lower firm-specific rate was the more relevant for analyzing a prospective merger, because merger-specific efficiencies should generally be viewed as firm-specific.(49)
The FTC's econometric expert, Dr. Orley Ashenfelter, working with the FTC econometrics team, developed a way to isolate empirically the firm-specific pass through rate.(50) The data set we employed included a measure of average variable cost by stock keeping unit ("sku")(51) and store for thirty products sold by both Staples and Office Depot.(52) Two regression models were estimated. The first related the Staples price to its own costs and fixed effects for store, sku, and time.(53) The coefficient on the Staples cost variable in this model was 0.57. Because variables were expressed in logarithms, this coefficient seemed to imply that when Staples' costs fell by 10%, it historically reduced price on average by 5.7% -- close to the two-thirds pass through rate suggested by the merging firms. But this historical average is not the right pass-through rate for analyzing the price effect of merger-specific cost savings because it combines the effects of industry-wide and firm-specific cost reductions.
To isolate the firm-specific pass-through rate, a measure of Office Depot costs was added to the regression model. Here, the Office Depot cost variable is thought of as a proxy for industry-wide costs -- after all, if costs fell for all firms in the industry, regardless of the market definition, they would surely fall for both of these firms.(54) With the Office Depot cost variable in the equation, the Staples cost variable would pick up only the effect of Staples-specific cost reductions. The results were striking: the Staples-specific pass-through rate was only 15%, much lower than the 57% figure suggested by our first model or the 67% figure claimed by the merging firms. In other words, if Staples costs fell 10% but its rivals' costs did not change, we found, Staples would lower price only 1.5%. As previously noted, Judge Hogan relied on this estimate in concluding that the cognizable efficiencies from the proposed merger would largely not be passed on to consumers.
III. Lessons
A. Lessons for antitrust policy
It is unlikely that the FTC would have brought the Staples case had the theory suggested by the documents not been confirmed with systematic empirical evidence. The anticompetitive theory had to overcome the natural presumption that defendants would be able to show that they were small players in a broad office supply retailing product market characterized by easy entry, and that they were merging merely to achieve greater scale economies more rapidly than internal growth would permit. Even though the party documents were inconsistent with this view, it was useful to confirm the anticompetitive theory with a systematic study of industry pricing. Moreover, we believed that our pricing studies undermined defendants' ability to rebut the evidence in their own pricing documents by asserting that the relationship the Commission alleged between Staples prices and rivalry from Office Depot was merely a "nonsense correlation" reflecting "cherry-picking" anecdotes.
While the result in Staples does not discourage the continued use of econometric studies of pricing (and cost pass-through rate), it does not mandate any specific form for the pricing analysis. In particular, future pricing studies may involve simulations based on reduced form price equations (the methodology employed by both sides in Staples), but they may instead involve simulations based on the estimation of demand elasticities. Reduced form price equations are attractive for expositing in court the systematic determinants of pricing because they relate price to market structure (concentration);(55) this methodology is sympathetic to the structuralist perspective of the case law. On the other hand, demand estimation is attractive because it is more sympathetic to the logic of the localized competition analysis central to the unilateral theory of adverse competitive effects of merger among sellers of differentiated products.(56)
B. Lessons about the use of econometric evidence
The FTC economic staff has conducted and reviewed econometric studies and simulations (predictions) derived from regression results in many merger investigations, including Staples. This experience gives rise to three observations about the process of reviewing econometric studies submitted to the Commission by the economists working with outside parties.(57)
The first observation grows out of a view of econometric analysis as a way of summarizing data. From this perspective, regression results presented by interested parties are an invitation to the FTC to interpret the data in a particular manner, much as briefs and white papers submitted by outside parties synthesize a view of the documentary and testimonial evidence. When interested parties quote documents and testimony in a brief or white paper, they are in effect asserting that if we go back to the original source material -- the full documents from which the quotes were selected, and the evidence not mentioned along with the cited evidence -- we will view the body of facts in the way they propose. That assertion is the most trustworthy when we have access to the documents and testimony that the parties reference, so we can see for ourselves the context in which statements were made, study internal indicia of credibility, and confirm key factual assertions.
This analogy suggests why it is important that interested parties submitting econometric studies make it possible for us to understand what they have done, reproduce it, and satisfy ourselves that results are not sensitive to alternative specifications. When outside parties submit regression results, they are in effect asserting that if we go back to the raw data, we will summarize it the same way they see it. That claim is the most credible when we have access to the raw data; understand how the data was collected and "cleaned";(58) understand which observations were included in the analysis; understand how variables used in the study (e.g. price indices) were created and transformed; understand how the regression model that relates the variables was specified; determine what statistical techniques were employed; study the full regression output (not just the coefficients of interest but also all estimated parameters, diagnostic statistics, and goodness of fit measures); understand how the regression results were interpreted (as bearing on the questions at issue in the investigation); and have the opportunity to "pressure test" those interpretations by reworking the study in our own way.(59) Thus, econometric studies and simulation analyses should receive little weight when submitted without the data, explanations, and other assistance we need to understand and replicate the parties' methodology in a timely manner.(60)
Sharing this information facilitates the development of a dialogue between the Commission staff and the parties about theory and evidence -- which we welcome. During a merger investigation, before a complaint has been issued, we routinely discuss our concerns with the merging firms, based on the documentary, testimonial, and empirical evidence we have reviewed.(61) Doing so helps us test possible theories, and it helps the firms identify additional evidence that might bear on our concerns.(62) This is manifestly in the staff's interest: we neither want to harm the economy by holding up procompetitive transactions nor learn about exculpatory evidence only after the Commission has decided to take a case to court.
From another perspective, econometric analysis is more than merely a way to summarize data. Econometric modeling almost necessarily requires methodological choices, including decisions about the measurement of variables; specification of functional form; assumptions about error structure; selection of an appropriate time period for the study (or other restrictions on what data to include); and choice of instrumental variables. This leads to a second observation: econometric analyses are more persuasive when key modeling choices are consistent with economic theory, informed by quantitative or qualitative information about the market, and tested against plausible alternatives. In the Staples litigation, for example, we preferred regression equations that included MSA-based competitor variables along with concentric circle variables, both because doing so was consistent with the documentary evidence about price zones and because the MSA-based variables contributed statistically to reflecting the intensity of competition. Similarly, we preferred simulations based on regression equations accounting for all the Staples stores to simulations based on regressions that excluded certain observations because we found the reasons defendants' expert offered for excluding the data unconvincing.(63)
The third observation about the process of evaluating econometric studies and simulations applies when the process for doing so is adversarial, regardless of whether the decision-maker is an enforcement agency deciding whether to challenge a merger or a court deciding whether to sustain such a challenge. In an adversarial setting, each party may present both its own analysis of the data and a criticism of the other side's analysis. Under such circumstances, the adversaries should be charged with assisting the decision-maker by narrowing the issues to those that matter to the ultimate conclusion. Thus, criticism of an econometric or simulation methodology should be treated with skepticism absent a demonstration that a reasonable alternative leads to a substantially different result, where such an analysis is possible.(64) In situations where the effect of the questioned methodology cannot be determined quantitatively, the party criticizing the other side's analysis should explain both why the other side's approach is inappropriate and why it is plausible that the difference between the inappropriate and preferred approaches is substantial. The FTC appealed to this principle in the Staples litigation in responding to the merging firms' criticism that it was inappropriate to pool observations nationwide when estimating a regression model. As previously noted, the Commission's econometric expert demonstrated that the nationwide simulation results were substantially similar -- indeed, more favorable to the Commission's position -- when the model was estimated regionally to address this criticism.