Gregory Mitchell, Laurens Walker & John Monahan, Beyond Context: Social Facts as Case-specific Evidence

BEYOND CONTEXT: SOCIAL FACTS AS CASE-SPECIFIC EVIDENCE

Gregory Mitchell* Laurens Walker** John Monahan***

ABSTRACT

Experts often seek to apply social science to the facts of a particular case. Sometimes experts link social science findings to cases using only their expert judgment, and other times experts conduct case-specific research using social science principles and methods to produce case-specific evidence. This Article argues against expert judgment as the means of linking general social science to specific cases, and for the use of methodologically rigorous case-specific research to produce “social facts,” or case-specific evidence derived from social science principles. We explain the many ways that social fact studies can be conducted to yield reliable case-specific opinions, and we dispel the view that litigation poses insurmountable barriers to the conduct of case- specific empirical research. Social fact studies are feasible for both plaintiffs and defendants, and they provide much sounder conclusions about the relevance of social science to a litigated case than does linkage via expert judgment.

* Mortimer M. Caplin Professor of Law & Class of 1948 Research Professor, University of Virginia.

Please direct correspondence to Greg Mitchell at University of Virginia School of Law, 580 Massie Road, Charlottesville, VA 22903-1738, greg_mitchell@virginia.edu. We appreciate the helpful input of Craig Callen, George Cohen, Brandon Garrett, Gregory Mandel, and workshop participants at Michigan State University College of Law and Temple University Beasley School of Law, and we appreciate the research assistance of Myra Chapman, Katherine Foster, Elizabeth Kade, Leah McLaughlin, and Adam Pollet.

** T. Munford Boyd Professor of Law, University of Virginia.

*** John S. Shannon Distinguished Professor of Law & Horace W. Goldsmith Research Professor of Law, University of Virginia.

INTRODUCTION 1111

1. SOCIAL FACTS DEFINED AND DESCRIBED 1115

   1. Comparing Social Facts to Social Authority and Social Frameworks 1116

   2. Types of Social Facts 1118

      1. Obtaining Descriptive Information 1118

      2. Obtaining Explanatory Information 1120

      3. Testing Specific Hypotheses 1124

2. THE ADMISSIBILITY OF SOCIAL FACTS 1128

   1. Internal Validity 1130

      1. Bias 1130

      2. Contamination 1131

   2. External Validity 1134

      1. Independent Variables: Do the Hypothesized Causal Variables in the Research Approximate the Causal Variables in the Plaintiff’s and Defendant’s Theories of

         the Case? 1135

      2. Dependent Variables: Are the Outcomes Studied in the Research Similar to the Outcomes at Issue in the Litigation? 1136

      3. Persons: Are the Research Participants Similar to the Persons Involved in the Case? 1138

      4. Setting: Are the Research Setting and Tasks Similar to Those in the Case? 1141

      5. Timing: Has Time Changed Anything? 1143

3. LEGAL AND ETHICAL ISSUES IN SOCIAL FACT STUDIES 1144

   1. Prefiling Research 1145

      1. Observational Studies 1145

      2. Surveys and Experiments 1146

      3. Discoverability 1148

   2. Postfiling Research 1149

      1. Social Facts Under the Federal Rules of Civil Procedure . 1150

      2. Ethical Issues 1154

CONCLUSION 1154

INTRODUCTION

Since Louis Brandeis first introduced it in court over a century ago,1 evidence drawn from social science research has played an important role in many forms of litigation.2 Perhaps the most prominent use of social science evidence occurs in employment discrimination class actions, where plaintiffs

often rely on the testimony of an expert to identify factors within the defendant organization that social science studies suggest could pose a common risk of harm to all class members. The prototypical example is found in Dukes v. Wal-Mart Stores, Inc., a nationwide gender discrimination class action

involving hundreds of thousands of women.3 A sociology expert reviewed the

case record on Wal-Mart’s employment practices in light of “what social science research shows to be factors that create and sustain bias and those that minimize bias”4 and concluded that Wal-Mart’s practices “contribute to disparities between men and women in their compensation and career trajectories at the company.”5

1. See Muller v. Oregon, 208 U.S. 412, 419 & n.1 (1908) (“[T]he brief filed by Mr. Louis D. Brandeis, for the defendant . . . . [included] extracts from over ninety reports of committees, bureaus of statistics, commissioners of hygiene, inspectors of factories, both in this country and in Europe, to the effect that long hours of labor are dangerous for women, primarily because of their special physical organization.”).

2. See generally JOHN MONAHAN & LAURENS WALKER, SOCIAL SCIENCE IN LAW (7th ed. 2010)

   (providing history and discussion of social science as used in a variety of legal contexts).

3. 603 F.3d 571 (9th Cir.) (en banc), cert. granted, 131 S. Ct. 795 (2010). The likely size of the class was a matter of some dispute between the majority and dissent in the en banc opinion issued by the Ninth Circuit.

   Id. at 578 n.3. For information about the proposed class and the relief sought, see Plaintiffs’ Third Amended Complaint at 21–25, Dukes v. Wal-Mart Stores, Inc., 222 F.R.D. 137 (N.D. Cal. 2004) (No. C-01-2252 MJJ), available at http://www.impactfund.org/documents/cat_95-100/Third_Amended_Complaint.pdf. Professor Nagareda describes Dukes as “the largest class action in history under Title VII of the Civil Rights Act of 1964.” Richard A. Nagareda, Class Certification in the Age of Aggregate Proof, 84 N.Y.U. L. REV. 97, 102 (2009) (footnote omitted).

4. Declaration of William T. Bielby, Ph.D. in Support of Plaintiffs’ Motion for Class Certification at 5,

   Dukes, 222 F.R.D. 137 (No. C-01-2252 MJJ), available at http://www.walmartclass.com/staticdata/ reports/r3.html.

5. Id. at 41. The district court relied on this expert’s opinions in granting the plaintiffs’ class certification

   motion. Dukes, 222 F.R.D. at 153–54, aff’d sub nom. Dukes v. Wal-Mart, Inc., 509 F.3d 1168 (9th Cir. 2007), aff’d in part on reh’g sub nom. Dukes v. Wal-Mart Stores, Inc., 603 F.3d 571 (9th Cir.) (en banc), cert. granted, 131 S. Ct. 795 (2010). The initial appellate panel also allowed the expert’s opinions as evidence of commonality. Dukes, 509 F.3d at 1179–80. The Ninth Circuit granted rehearing en banc, and a split court upheld the district court’s certification decision. Dukes, 603 F.3d at 628. We discuss in greater detail the opinions of the expert in Dukes—and our concerns about those opinions—in two recent publications. See John Monahan, Laurens Walker & Gregory Mitchell, Contextual Evidence of Gender Discrimination: The Ascendance of “Social Frameworks,” 94 VA. L. REV. 1715, 1742–48 (2008) [hereinafter Monahan et al., Ascendance of Social Frameworks] (arguing that the expert, although claiming to present a social framework, testified about social facts specific to the defendant); John Monahan, Laurens Walker & Gregory Mitchell, The Limits of Social Framework Evidence, 8 LAW, PROBABILITY & RISK 307, 311–19 (2009) [hereinafter Monahan

   The approach the expert used in Dukes has come to be known as “social framework analysis”—so called because an expert uses social science research as a framework for analyzing the facts of a particular case.6 In this approach, the expert uses her judgment, rather than traditional empirical methods, to link social science propositions to a particular case.7 For instance, the expert in Dukes conducted no observational, statistical, or experimental tests to determine whether any particular employment practice of Wal-Mart actually

   contributed to sex disparities in pay; he simply reviewed discovery materials and judged Wal-Mart’s practices to contain features that social science studies suggest can be associated with intergroup bias.8 This method has been

   
   

   et al., Limits of Social Frameworks] (noting a lack of objective measurements to ensure facts were true and representative of all of defendant’s locations).

6. Professors Susan Fiske and Eugene Borgida first used this phrase as an extension of Walker and

   Monahan’s “social frameworks” concept, which involves using general social science evidence to provide a frame of reference or background information to assist the factfinder deciding issues in a specific case. Compare Susan T. Fiske & Eugene Borgida, Social Framework Analysis as Expert Testimony in Sexual Harassment Suits, in SEXUAL HARASSMENT IN THE WORKPLACE: PROCEEDINGS OF NEW YORK UNIVERSITY

   51ST ANNUAL CONFERENCE ON LABOR 575, 577 (Samuel Estreicher ed., 1999) (“A social framework analysis uses general conclusions from tested, reliable, peer-reviewed social science research and applies it to the case at hand.”), with Laurens Walker & John Monahan, Social Frameworks: A New Use of Social Science in Law, 73 VA. L. REV. 559, 559 (1987) (“[G]eneral research results are used to construct a frame of reference or background context for deciding factual issues crucial to the resolution of a specific case. We call this . . . social frameworks.”). Whereas Walker and Monahan envisioned judicial instructions on reliable social science propositions that would provide jurors with new general information to help them make sense of the case-specific evidence, Fiske and Borgida described a “newer methodology of social framework analysis” in which “[c]onclusions aggregated from the research literature are applied to particular cases” by expert witnesses. Fiske & Borgida, supra, at 575–77. As we have discussed previously, in our view case-specific opinions based on Fiske and Borgida’s “social framework analysis” violate basic rules of expert evidence and scientific reliability and should not be confused with Walker and Monahan’s conception of social framework evidence. See Monahan et al., Ascendance of Social Frameworks, supra note 5, at 1746 n.84 (“Whereas Walker and Monahan expressly argued that any inferences to be drawn from the general research to the specific case should be the province of the fact-finder working within a court’s instructions, Fiske and Borgida expressly advocated that experts make such linkages for the fact-finder . . . .”); Monahan et al., Limits of Social...