EDRM At Duke Law School Releases TAR Guidelines

Demystifying the technology assisted review process and providing some guidance on how it can be applied to document review in discovery.

One of the big buzzwords of late is “innovation.” Law firm leaders are talking about chief innovation officers and corporate legal operations people are not only looking for firms that are technologically innovative, but also at how they themselves might innovate. This week, we cover a topic that is not overly innovative, but one that just makes perfect common sense.

Last week, the EDRM at Duke Law School released its Technology Assisted Review (TAR) Guidelines (full disclosure: I am among the members of the drafting and editing team at EDRM). When we started the TAR Guidelines project, we really only had one mission: To promote and foster greater use of TAR. There wasn’t any discussion about TAR being particularly innovative. I mean, maybe 15 years ago TAR was innovative. But today? And in writing the TAR Guidelines we did not set out to invent a new platform or process. Instead, we simply asked what is inhibiting wider use of TAR?

We talked about a lot of things in this regard, including the cost, the learning curve, the size of the case, and even whether the outcomes of TAR projects would be accepted by the parties and the courts. But while these are or were real obstacles to adoption, they have mostly been overcome over the past decade. I’ve always thought that TAR is not more broadly used simply because people don’t understand it. So, we set about to demystify the process, provide some guidance, and suggest how it may be applied to document review in discovery.

It is helpful, I think, to illustrate the utility of TAR in basic terms. Before TAR, an attorney looking at documents in discovery would review each document and make a judgment on whether the document was relevant or not. Words and phrases or concepts in the documents inform that judgment. After reviewing each document, the relevant documents are put in one pile; the non-relevant documents in another pile. This manual process required teams of attorneys and was very time consuming and costly. Imagine using this manual process to review a million documents.

Now imagine using TAR to achieve a similar outcome. A typical workflow requires that the documents are ingested into a computer. The text is extracted from each document and captured by the computer in an index. A machine learning algorithm then analyzes the content of each document, the relationship between words, phrases and characters, the frequency and pattern of terms and other features and characteristics of the documents. The algorithm forms a conceptual model or understanding of the content of each document. Next, in exactly the same way that they make judgments during manual review, attorneys examine a fraction of the documents and mark or code this subset of documents as either relevant or non-relevant. From the relevancy classifications made by the attorney, the machine learning algorithm is able to learn the attorney’s preferences. It learns conceptually what constitutes a relevant document and what constitutes a non-relevant document. Then, using the conceptual model built on each document and the input from the reviewing attorney, the algorithm applies the human reasoning to conceptually similar documents to classify the documents that have not been reviewed by the attorney. That’s TAR in a nutshell.

The difference between manual human review and TAR — and ultimately, the benefit to the legal profession and clients — is that machine learning programs are capable of making relevancy judgments much more quickly and consistently than humans. This saves not only time, but a great deal of money. Additionally, time is not wasted looking at most of the non-relevant documents.

Most people don’t realize that machine learning, which is at the core of every TAR software available in the legal industry, has been in commercial use since the 1950s. Machine learning has been used in finance, in marketing and in data security. It is used in healthcare to diagnose and treat illness, and of course machine learning is literally driving the development of autonomous vehicles. TAR, or predictive coding as some call it, is just the legal industry version of machine learning.

Sponsored

When we talk about innovation, we seem to focus on the latest tool or platform that automates some task, makes a mundane exercise easier, faster, or cheaper, or brings together data points that give insight to previously elusive conclusions. And there’s a lot of this going on in the legal space right now. And that’s great. TAR may not be particularly innovative as a technology, but the idea behind it, the processes which are employed, and the decision to use it, can indeed be quite innovative.

Honestly, if I were in legal operations or the GC responsible for the legal budget, I cannot imagine not using machine learning technologies to save time and money in discovery.

The EDRM’s TAR Guidelines consist of four chapters. The first chapter defines technology assisted review and outlines at a high level the TAR process. The second chapter lays out a suggested workflow for the TAR process. The third chapter examines alternative use cases for TAR. Chapter four discusses factors to consider when deciding whether to use TAR. The TAR Guidelines are available for download here.


Mike Quartararo

Sponsored

Mike Quartararo is the managing director of eDPM Advisory Services, a consulting firm providing e-discovery, project management and legal technology advisory and training services to the legal industry. He is also the author of the 2016 book Project Management in Electronic Discovery. Mike has many years of experience delivering e-discovery, project management, and legal technology solutions to law firms and Fortune 500 corporations across the globe and is widely considered an expert on project management, e-discovery and legal matter management. You can reach him via email at mquartararo@edpmadvisory.com. Follow him on twitter @edpmadvisory.

CRM Banner