The Uncertain Promise of Predictive Coding

The Uncertain Promise of Predictive Coding Dana A. Remus  “Technology . . . is a queer thing; it brings you great gifts with one hand, and it stabs you in the back with the other.” – C.P. Snow, 1971 1 ABSTRACT: Increasingly, machine-learning technologies known as “predictive coding” are automating document review in discovery practice. Recent law school graduates may lament the impact on entry-level law hiring, but the litigation community is embracing the new technologies. Proponents contend that by replacing the unreliable and inconsistent discretion of lawyers with the mechanized objectivity of computers, predictive-coding technologies can solve both the practical problems of e-discovery and the deeper-seated problems of excess, abuse, and trust that have long plagued discovery practice. In this Article, I advise caution in the adoption of predictive-coding technologies. These technologies hold unquestionable potential as a means of coping with unmanageable datasets, but they entail costs as well as benefits. I argue that if lawyers ignore these costs, they will unwittingly abdicate control to computer scientists and vendors, compromising the profession’s jurisdiction and undermining lawyers’ ability to serve clients and the judicial system. I conclude that the profession has an ethical obligation to explore the costs as well as the benefits of predictive coding, and to play a more active role in its design and use.  Associate Professor of Law, University of North Carolina School of Law. For helpful comments on earlier drafts, I am grateful to David Ardia, Bernard Burke, John Coyle, William Marshall, Mary Mitchell, Richard Myers, Mary-Rose Papandrea, Richard Saver, Robert Smith, W. Bradley Wendel, and the participants of the 2013 Fordham Legal Ethics Schmooze. For outstanding research assistance, I am grateful to Christina Nasuti, Troy Shelton, and Alex Shiekman. 1. Anthony Lewis, Dear Scoop Jackson , N.Y. TIMES, Mar. 15, 1971, at 37 (quoting C.P. Snow). 1692 IOWA LAW REVIEW [Vol. 99:1691 INTRODUCTION .................................................................................... 1693 I. A NEW APPROACH TO DISCOVERY ........................................................ 1695 A. C IVIL D ISCOVERY P RACTICE ............................................................ 1696 B. E-D ISCOVERY .................................................................................. 1698 C. P REDICTIVE C ODING ....................................................................... 1701 II. THE CONSEQUENCES OF ADOPTION .................................................... 1706 A. V ARIATION AND U NCERTAINTY ....................................................... 1706 B. T HE P ROFESSION ’ S J URISDICTION .................................................... 1708 1. The Unauthorized Practice of Law ..................................... 1708 2. Control over Court Processes .............................................. 1710 3. Patent Monopolies ............................................................... 1712 C. A DVERSARIAL V ALUES ..................................................................... 1715 1. New Kinds of Cooperation .................................................. 1716 2. New Degrees of Cooperation .............................................. 1717 III. THE PATH AHEAD ................................................................................ 1719 A. E DUCATION .................................................................................... 1719 B. Q UALITY C ONTROL ........................................................................ 1720 C. S TANDARDIZATION ......................................................................... 1721 D. A CCESS .......................................................................................... 1722 CONCLUSION ....................................................................................... 1723 2014] THE UNCERTAIN PROMISE OF PREDICTIVE CODING 1693 INTRODUCTION Problems of adversarial excess and abuse have long plagued civil discovery. In 1976, a commission convened by Chief Justice Burger concluded that “[w]ild fishing expeditions” had become the norm, along with “[u]nnecessary intrusions into the privacy of the individual, high costs to the litigants, and correspondingly unfair use of the discovery process as a lever toward settlement.” 2 Today, nearly forty years later, the ethical and practical problems of discovery practice have only worsened and all too often, lawyers appear to be part of the problem. When the drafters of the Federal Rules of Civil Procedure instituted civil discovery in 1938, they envisioned a largely self-regulating system, entrusted to the sound professional judgment and discretion of lawyers. 3 Lawyers, they believed, would act as professionals in discovery practice, balancing their tripartite duties to clients, courts, and the public. But lawyers quickly proved themselves to be adversarial advocates first and foremost, prioritizing their clients’ interests above all else. Hidden from public view and with limited accountability, they turned discovery practice into a new area of gamesmanship, 4 which, in turn, undermined trust in the court system and the legal profession. 5 The advent of computer technology and the proliferation of electronically stored information layered a new set of problems on top of existing ones. 6 Companies faced new and expensive questions regarding document retention, preservation, and production. Litigants began “document dumping”—flooding opponents with unmanageable datasets to increase costs and decrease their chances of finding key documents. The unmanageable scope and extent of e-discovery offered new opportunities for abuse and quickly became a principal cause of increasing costs and delays in the court system. 2. William H. Erickson, The Pound Conference Recommendations: A Blueprint for the Justice System in the Twenty-First Century , 76 F.R.D. 277, 288 (1978). 3. See Stephen N. Subrin, Fishing Expeditions Allowed: The Historical Background of the 1938 Federal Discovery Rules , 39 B.C. L. REV. 691, 717 (1998) (quoting Edson R. Sunderland, Improving the Administration of Civil Justice , ANNALS AM. ACAD. POL. & SOC. SCI., May 1933, at 60, 76). 4. See John S. Beckerman, Confronting Civil Discovery’s Fatal Flaws , 84 MINN. L. REV. 505, 522 (2000) (“Moreover, to the extent that success in litigation depends on strategic informational advantage, discovery, contrary to its inventors’ expectations, is the critical battlefield on which the war is waged.”). 5 . See id. ; John H. Beisner, Discovering a Better Way: The Need for Effective Civil Litigation Reform , 60 DUKE L.J. 547, 549 (2010); Louis Harris & Assocs, Inc., Judges’ Opinions on Procedural Issues: A Survey of State and Federal Trial Judges Who Spend at Least Half Their Time on General Civil Cases , 69 B.U. L. REV. 731, 733, 735–36 (1989) (reporting that of 1000 surveyed federal and state judges, many believed that discovery abuse was “the most important cause of delays in litigation and of excessive costs”). 6. See Beisner, supra note 5, at 550 (“The exponential growth in the volume of electronic documents created by modern computer systems has exacerbated the problem of abusive discovery and is jeopardizing the legal system’s ability to handle even routine matters.”). 1694 IOWA LAW REVIEW [Vol. 99:1691 Against this backdrop, bar leaders and reformers are now advancing a new solution to the problems of discovery. 7 Rather than advocating trust in the profession, they are advocating trust in computers—more specifically, in machine-learning products referred to as “predictive coding.” 8 Although predictive-coding technologies encompass significant variations, they share a common approach: after an initial training period, a computer generates a customized search algorithm for identifying responsive and privileged documents; the computer then uses the algorithm to code an entire document set for responsiveness and privilege, obviating the need for manual human review. 9 Proponents frame predictive coding as a silver-bullet solution to the problems of discovery practice—not only the practical problems of scope and cost, but also the more vicious problems of excess, abuse, and trust that have long characterized discovery practice. They claim that by eliminating the time and inconsistency of human review, predictive coding can increase the accuracy and decrease the costs of document review; and by replacing human discretion with mechanized objectivity, it can minimize abuse and restore trust in the system. 10 Predictive coding holds far more potential, they argue, than continued efforts to regulate attorney conduct. Since 2012, a handful of trial courts have accepted these arguments, officially endorsing predictive coding as a valid and promising means of meeting discovery obligations. 11 The litigation community followed this vanguard and embraced the new technology, 12 such that predictive coding is now the “hot topic” of discovery reform. 13 Most large law firms have in-house 7. See infra notes 60–65 and accompanying text. 8. See EDISCOVERY INSTITUTE SURVEY ON PREDICTIVE CODING 2 (2010), [hereinafter EDISCOVERY SURVEY], available at http://www.discovia.com/wp-content/uploads/2012/07/ 2010_EDI_PredictiveCodingSurvey.pdf. 9. See infra notes 49–55 and accompanying text. 10. See infra notes 59–61 and accompanying text. 11 . See Da Silva Moore v. Publicis Groupe, 287 F.R.D. 182 (S.D.N.Y. 2012), adopted sub nom Da Silva Moore v. Publicis Groupe SA, No. 11 Civ. 1279(ACL)(AJP), 2012 WL 1446534 (S.D.N.Y. Apr. 26, 2012); Case Management Order, In re Actos (Pioglitazon—Prods. Liab. Litig.), No. 6-11-md-2299, 2012 WL 3899669 (W.D. La. July 30, 2012); Order Approving the Use of Predictive Coding for Discovery, Global Aerospace, Inc. v. Landow Aviation, L.P., No. CL 61040, 2012 WL 1431215 (Va. Cir. Ct. Apr. 23, 2012); Order Granting Partial...