Use of a WOE methodology may be appropriate for government regulation, but it should not establish legal liability.
The article was published in the Winter 2017 issue of In-House Defense Quarterly, a publication of DRI. It is reprinted here with permisson.
"Because I say so" is not a reliable scientific methodology. But plaintiffs’ counsel have, with some success, invoked "weight of the evidence" (WOE) methodology—a process used by some regulatory bodies to classify theoretical hazards—in an effort to mask their experts’ "say so" approach. And some courts have gone along. But WOE methodology has no legitimate place in the courtroom.
Regulators do not shoulder the burden borne by plaintiffs. They use WOE to alert the public to possible hazards. Legal factfinders, on the other hand, impose liability in cases where the evidence establishes that a product more likely than not caused an injury. Given these different goals, the fact that regulators may use WOE methodology to evaluate scientific data does not earn it admission in civil litigation.
We believe that decisions admitting expert testimony based on WOE are wrong, but do not repeat the reasons why here, as they have been discussed extensively by others. See, e.g., David E. Bernstein, The Misbegotten Judicial Resistance to the Daubert Revolution, 89 Notre Dame L. Rev. 27 (2013) (criticizing the First Circuit’s endorsement of the WOE approach as an example of "judicial noncompliance" with Federal Rule of Evidence 702 and Daubert); see also Jennifer L. Mnookin, Atomism, Holism, and the Judicial Assessment of Evidence, 60 UCLA L. Rev. 1524, 1580 (2013) ("To be sure, methods for aggregation in science, even relatively informal, weight-of-the-evidence approaches certainly ought not to be based on the expert’s mere say so."). Instead, after reviewing the conflicting case law, we offer practical approaches for challenging experts who use WOE to offer ipse dixit testimony.
WOE Methodology: Conflicting Definitions
"Weight of the evidence" is a common phrase without a clear meaning. The phrase has been defined as "a process or method in which all scientific evidence that is relevant to the status of a causal hypothesis is taken into account." Sheldon Krimsky, The Weight of Scientific Evidence in Policy and Law, 95 Am. J. Public Health (Supp. 1) S129 (2005). In practice, WOE is most often used as a metaphorical term for a subjective assessment of "relevant" data examined for some risk or hypothesis, without reference to any interpretive methodology. Douglas L. Weed, Weight of Evidence: A Review of Concept and Methods, 25 Risk Analysis 1545, 1546–47 (2005); see also Krimsky, supra, at S129. But it may also be used to describe a methodological approach that could include systematic reviews, quality criteria for toxicology studies, causal criteria in epidemiology, meta-analysis, mixed epidemiologic-toxicology models and quantitative weighting schemes. Weed, supra, at 1547–52; see also Krimsky, supra, at S129.
The variability in WOE characterizations leads to wide variability in how scientists and regulatory agencies exercise judgment under WOE. "Metaphorically, judgment is a kind of intellectual glue, cementing together the evidence and the methods." Weed, supra, at 1553. "Without an explication of how evidence is ‘weighed’ or ‘weighted,’ the claim WOE seems to be coming out of a ‘black box’ of scientific judgment." Krimsky, supra, at S131 (quoting M.A. Ibrahim, et al., Weight of the Evidence on the Human Carcinogenicity of 2,4-D, 96 Envtl. Health Perspectives 213 (1991)).
WOE Regulatory Methods Are Problematic in a Courtroom
Use of a WOE methodology may be appropriate for government regulation, but it should not establish legal liability. As the Reference Manual explains:
The agency assessing risk may decide to bar a substance or product if the potential benefits are outweighed by the possibility of risks that are largely unquantifiable because of presently unknown contingencies. Consequently, risk assessors may pay heed to any evidence that points to a need for caution, rather than assess the likelihood that a causal relationship in a specific case is more likely than not.
Reference Manual on Scientific Evidence, Fed. Jud. Ctr., at 33 (2d ed. 2000).
Following this logic, courts have appropriately recognized that the U.S. Food and Drug Administration (FDA) utilizes a much lower standard of proof for taking regulatory action than that applied by a court to determine causation. See, e.g., Allen v. Pennsylvania Engr’g Corp., 102 F.3d 194, 198 (5th Cir. 1996) (rejecting experts’ reliance on the methodology employed by regulatory agencies because "[t]he agencies’ threshold of proof is reasonably lower than that appropriate in tort law, which ‘traditionally make[s] more particularized inquiries into cause and effect’ and requires a plaintiff to prove ‘that it is more likely than not that another individual has caused him or her harm’"). Similarly, WOE regulatory risk assessments used by federal, state, and international agencies to evaluate potential health risks have also been rejected by courts as proof of causation. See, e.g., Abarca v. Franklin Cnty. Water Dist., 761 F. Supp. 2d 1007, 1040-41 (E.D. Cal. 2011) (granting defendants’ motion for partial summary judgment where plaintiffs relied on an EPA Risk Assessment to establish causation); Rhodes v. E.I. DuPont de Nemours & Co., 253 F.R.D. 365, 377–78 (S.D.W.Va. 2008) ("Because a risk assessment overstates the risk to a population to achieve its protective and generalized goals, it is impossible to conclude with reasonable certainty that any one person exposed to a substance above the criterion established by the risk assessment has suffered a significantly increased risk").
Nonetheless, given the many definitions of WOE and different uses of WOE in the regulatory public health context, courts have reached conflicting results in deciding the admissibility of opinions based on WOE.
WOE Methodology in the Courtroom
WOE Cases: Conflicting Decisions
Federal Courts
Some federal courts have found that, when "properly applied, the weight-of-the-evidence methodology is not an unreliable methodology." Magistrini v. One Hour Martinizing Dry Cleaning, 180 F. Supp. 2d 584, 602 (D.N.J. 2002), aff’d, 2003 U.S. App. LEXIS 12972 (3d Cir. June 25, 2003) (expert excluded). See also Milward v. Acuity Specialty Prods. Group, 639 F.3d 11 (1st Cir. 2011) (reversing exclusion of expert testimony based on WOE methodology); Allen, 102 F.3d at 197 (excluding experts who used a WOE methodology); Waite v. AII Acquisition Corp., No. 15-cv-62359, 2016 U.S. Dist. LEXIS 107820, *35 (S.D. Fl. July 11, 2016) (finding WOE methodology employed was "sound"); In re Chantix, 889 F. Supp. 2d 1272, 1293 (N.D. Ala. 2012) (permitting expert testimony relying on WOE methodology).
In Magistrini, the court excluded the plaintiffs’ general causation expert testimony because the expert’s application of WOE methodology did not explain his rejection of studies that failed to support his conclusions. Id. at 603. There, the plaintiff claimed that her occupational exposure to a dry cleaning chemical known as perchloroethylene caused her to develop acute myelomonocytic leukemia. Id. at 589. The plaintiff’s general causation expert, Dr. David Ozonoff, cited the studies that he considered, but he did not explain "the methodology he...
