The Truth Matters in E-Discovery

Lady justice, blind and weighing the truthThe truth definitely matters. We’ve all read horror stories from the criminal courts. Unqualified experts, bad identification evidence, perjured testimony, tainted DNA, corrupt cops, less than candid prosecutors–all these things lead to unreliable outcomes in the mostly analog world of criminal jurisprudence. It’s no different in civil litigation when it comes to the use of electronic evidence. Information that companies possess and control can be manipulated, altered, corrupted, or deleted—inadvertently or deliberately. In most instances, there’s nothing nefarious going on, but occasionally parties and lawyers are sanctioned for wrongdoing in civil litigation as well.

Regardless of the forum, the expectation is that the information put before judicial factfinders, hearing officers, and others, is complete and accurate. One would like to think that good, accurate information leads to the “truth and justice” in our system of jurisprudence. Because, at the end of the day, the truth matters in both civil and criminal cases and fact-finders simply cannot get to the truth if the information presented is bad.

Information is Everywhere

But the fact is that the pursuit of truth is more complex today because we live in a world in which almost everyone is surrounded by vast amounts of electronically stored information (ESI). The Radicati Group estimates that the business world accounts for more than 108 billion emails sent and received per day. They estimate that number will to grow to 140 billion by 2018.[1]

And there is more information created in the world in the last few years than exists throughout all of humankind in the thousands of years prior. Ubiquitous computer devices transmit and store information. Our locations, up-to-the-minute news, sound bites, email, texts, and social media feeds, not to mention “Alexa,” new doorbells, and smartphone-controlled appliances. All this information may play a role in the truth-seeking process. It can help litigants to investigate events and tell their stories in court.

Information is now everywhere and nearly everything we do today is dependent upon electronic devices that store information. These machines are integral to daily life and they supplement the shortages of mere mortal memories. Indeed, computers now perform some functions that previously only humans performed. And computers are now smarter with technologies like Watson’s augmented intelligence and machine learning.

Still, there is work to be done.

The E-Discovery Market

In the United States, which is 80% of the global market, electronic discovery is a growing and thriving field. The market worth of the global e-discovery industry doubled since 2010 and the projection is it will quadruple by 2020. Driven by the massive increase of ESI and the need to manage that information for civil litigation, the market, including services and software, grew to over $7 billion in 2015.[2] One report projects 16% compound annual growth rate for services and software through 2022, increasing the market to more than $20 billion.[3] A Gartner report similarly projects double-digit year-over-year growth in e-discovery software.[4]

At the same time, due to the recession and slow economic recovery, the legal services industry is undergoing considerable change. Consider that law firms are folding and merging or reducing personnel to stay afloat. Corporate legal departments face pressure to reduce legal expenses. One need not have an MBA or a law degree to conclude that the current business model at many firms may not be sustainable in the long term. To survive, lawyers and legal support staff need to change the way they practice and deliver legal services.

By far, the most costly and time consuming aspect of litigation is the discovery process. The Rand Corporation reports that 75% of expenditures in litigation are attributable to discovery costs.[5] It used to be that junior lawyers would sit in rooms filled with boxes to review documents. Discovery is no longer about boxes of paper; instead, it is about terabytes of data. And discovery costs are higher not because of inflation or because attorney hourly rates have risen. In fact, e-discovery costs are considerably lower today than 15 years ago. No, discovery is more expensive now because there is so much more information and the higher volume of ESI drives that cost.Truth and justice at the Supreme Court The tsunami of information that has washed over the practices of the world’s largest organizations is not receding.

The Federal civil rules, adopted in 1934 and infrequently revised since then, have been amended twice since 2006 specifically to address ESI. The number of reported cases relating to e-discovery has skyrocketed recently and continues to grow. There are now e-discovery professional organizations and a multi-billion dollar e-discovery software and service provider industry capitalizing on the changes in legal practice and the deluge of ESI. Recruiters and headhunters now specialize in placing project managers, technical analysts, e-discovery specialists, and yes, e-discovery attorneys.

Going from Good to Great

So, the question becomes how to distinguish between organizations that are managing e-discovery projects well and those who are not. The solution, I propose, lies in identifying those organizations that have a consistently structured process for managing e-discovery more effectively and efficiently. I’m talking, of course, about organizations that apply project management principles to discovery projects.

There has been considerable discussion regarding the applicability of project management in a legal setting. Driving the debate is the need of law firms and corporate legal departments to find efficiencies and reduce legal expenses. Law firms, which rely upon corporations for a slice of the $300 billion legal services market, have begun to listen. Firms are adjusting rates, staffing leanly, or entering unique and tailored billing agreements. Few firms, however, are dramatically changing the ways in which lawyers work. And fewer still have adopted project management principles into their business model.

Applying project management methodologies to e-discovery projects is one tool that will differentiate great firms from good firms. And it will provide the efficiency and sustainability needed in a legal market that is very different today.

Bringing Order to Chaos

Project management brings structure and common business sense to law firms, which traditionally do not operate like a business. To the majority of people in the world –even the legal business—the words “electronic discovery” have little or no meaning. But “project management” is intuitive and people understand it to mean leading people, marshaling resources, and managing processes that lead to a desired outcome.

Most people have an inherent if not instinctive desire to be organized and efficient. It is why we categorize almost everything; it is frankly how the human mind works. Some people do it better than others, for sure. But in the end, we all want to get from point A to point B by the most direct route. And this means completing a task in the most efficient manner possible.

The use of planning, budgeting, and scheduling techniques, if executed correctly, can only enhance the discovery process. Ultimately, this aids the delivery of professional legal services. Monitoring and controlling e-discovery processes are critical as well. To maintain quality and effectively manage changes in scope, timing or cost, it is necessary to monitor each phase of an e-discovery project. And closing a discovery project is equally important. Performing a post-project review, archiving project documentation, and recording important metrics serve to prove a defensible process. They also provide much needed information for estimating The gavel of wtruth and justicefuture analogous projects.

The use of project management in electronic discovery can end the self-inflicted wound the legal industry imposed upon itself. Just like the record industry missed the boat on digital content, much of the legal industry missed the e-discovery boat. In many cases, it sailed right by.

Project management simply brings order to the chaos and puts the focus on sound, defensible processes. With project management, parties to litigation have reasonable assurance that managing their ESI is consistent with winning strategies. And this can help parties present their case in court. Quite apart from the cost, if missteps occur in the process it may be difficult to present good, accurate and complete information to the fact-finder. Following a consistent, structured process can only enhance a litigant’s chance of success.

Conclusion

Discovery of the truth is a foundational principle of our civil and criminal justice systems. When courts render judgment in a legal proceeding we want to believe that the truth has enlightened the path to justice. The judicial truth-seeking process, however, is only as good as the information presented as evidence. If you present unreliable information, the truth is compromised. It has now become clear that using project management principles can and will serve law firms and their clients well. An added bonus is the improvement of the truth-seeking process.


[1] The Radicati Group, Email Statistics Report 2014-2018.

[2] E-Discovery Market by Solution, Service, Deployment, Industry, & Region—Global Forecast to 2020 (Research and Markets, July 2015). http://www.researchandmarkets.com/research/zhg5cn/ediscovery  (global e-discovery software and services market expected to grow from $7.01 billion in 2015 to $14.2 billion in 2020—CAGR of 15.3%).

[3] eDiscovery Market–Global Industry Analysis, Size, Share, Growth, Trends and Forecast 2014-2022 (Transparency Market Research, 2015), http://www.transparencymarketresearch.com/ediscovery-market.html.

[4] , Zhang & Landers, Magic Quadrant for E-Discovery Software, Market Overview (Gartner, 2015), http://www.gartner.com/technology/reprints.do?id=1-2G57ESF&ct=150519&st=sb (e-discovery software market grew to $1.8 billion in 2014 and estimating five year CAGR of 12% with growth to more than $3 billion by 2019).

[5] Where the Money Goes: Understanding Litigant Expenditures for Producing Electronic Discovery (Rand Corporation, Institute for Civil Justice 2014).


Michael Quartararo is the firm-wide Director of Litigation Support Services at New York-based Stroock & Stroock & Lavan LLP and the author of the book Project Management in Electronic Discovery, published in June 2016 by eDiscoveryPM.com, LLC. He is a graduate of the State University of New York and he studied law in the UK. He is a certified Project Management Professional (PMP) and a Certified E-Discovery Specialist (CEDS). This article is adapted from content first published in Project Management in Electronic Discovery.

Alternative Facts and Artificial Intelligence

A few months back, a post here began by asking “Who Put the ‘Artificial’ in Artificial Intelligence?” At the time, I was sort of complaining about low adoption rates for technology tools in legal. I suggested that good marketing and “shiny object syndrome” are responsible for the rise of new technologies in legal. In the end, I question whether it’s appropriate to describe technology tools in the legal business as “artificial intelligence” tools.

Generally, I hardly think of myself as an “I told you so” type of person. If I’m being honest, though, I do think I told you so more than I articulate I told you so. (I like to think I’ve learned something tactful over the years). But an interview on Fortune.com got me to thinking.

IBM Logo Artificial IntelligenceFortune spoke to David Kenny, leader of the Watson Group at IBM. I thought while reading the interview that if anyone has a good grasp of artificial intelligence it’s IBM, right? Well, I have news: the word “artificial” really has no place in the technology tools that are in use in the legal industry—at least not the ones in use in e-discovery.

It’s Elementary My Dear Watson

To distinguish between the many AI terms out there, David Kenny said,

“Deep learning uses more advanced things like convolutional neural networks, which basically means you can look at things more deeply in more layers. Machine learning could work, for example, when it came to reading text. Deep learning was needed when we wanted to read an X-ray. And all of that has led to this concept of artificial intelligence—though at IBM, we tend to say, in many cases, that it’s not artificial as much as it’s augmented. So, it’s a system between machine computing and humans interpreting, and we call those machine-human interactions cognitive systems.”

“As for what we would call unsupervised learning—which is to say, we’re not training it to process but it’s beginning to learn on its own—that is moving more in the direction of what some consider true artificial intelligence, or even AGI: artificial general intelligence. I would say we’re at the early stages of that.”

Explaining how Watson is able to ingest the text of 26 million medical and scientific articles to help doctors find clinical trials for cancer patients, Mr. Kenny said:

“. . . it starts with knowledge extraction: reading documents, finding common phrases, associating those together. It does the same with paragraphs. Then it has to get corrected. The human annotation is critical here: Out of the gate there’s no way that I would trust the system to do unsupervised learning and just find the patterns on its own. You literally tell Watson, “Yes, that meant this, yes those go together. Yes, you have that right, or, no, you don’t.”

“And when you tell the system the “no’s,” it re-weights its algorithms until it gets to a point where it would have produced the correct answer. And it gets better over time.”

True Artificial Intelligence: Who’s Supervising Who

There you have it, folks. That is machine learning, hands down. And it’s precisely how the technology applies in e-discovery. It’s not a stretch, then, to say that what we are currently doing in e-discovery is not artificial intelligence. Additionally, what many now refer to as artificial intelligence is actually machine learning. Or, for the marketing folks out there, augmented intelligence (it’s still “AI” but not quite as sexy, right?)

To call it anything else, I think, is irresponsible and misleading. Lest we forget –and unless I missed something—the TAR, predictive and analytic programs in use in legal are algorithmic tools that require some level of human input. That’s not AI, as Kenny points out, because true AI is not supervised.

Why Does This Matter?

It matters because I think it’s not accurate. The reason it is “augmented” and not “artificial” in its application in the legal business is fairly obvious. But in addition, as far as I know, there are no lawyers –none who want to keep their licenses anyway—who are making important decisions, giving advice to clients, or reviewing and producing documents based on true artificial intelligence. No one plugs documents into a computer and spits them out without lawyers being involved (at least I hope not). Seems to me that there’s always a human to some degree augmenting the process.

Now, I’m not a scientist or a linguist. Heck, I barely qualify as a writer. It’s troubling, though, that in an industry in which facts and a search for the truth are paramount too much time and money goes to promoting products and ideas that are, how should I say, “artificial.”

I think it was the first celebrity law enforcement officer, Joe Friday, who said “Just the facts Ma’am.” So, let’s just stick to the facts in e-discovery and try not to promote tools that promise the stars and regrettably fall short. Ultimately, I think doing so will help foster adoption of technology in the legal industry.

I’d love to hear your thoughts on the issue, especially those of the scientists out there.

Can 650,000 Emails Be Reviewed In Eight Days?

workflowheaderIn the email debacle that engulfed the presidential election this past week, the question arose whether the government could possibly review 650,000 emails in eight days. With today’s technology, not only is it possible, but those who work in the litigation/practice support business know full well how it might be done. What surprises me I guess is that more people don’t realize that it’s possible.

My friend Eric Mandel recently wrote on LinkedIn about what could be learned in just 72 hours about the new emails. And I commented that I don’t understand how we in the industry could understand this, but the government does not. Well, apologies to the government – apparently they were doing (had to be doing) what we suspected all along –using technology to churn through the emails to determine whether there was anything of substance there.

We Have the Technology

For the uninitiated, using technology available today –technology, by the way that has been available for a long time— the government could not only parse through the email on that laptop in no time at all, they could also gain some real insight into the substance of the documents. To conclude otherwise, would be to deny science (and that’s not something politicians do, is it?).

How did they do this? As Eric pointed out, the initial question is whether this new tranche of emails contains anything new. One might start by doing some basic filtering and culling. Eliminate junk email, parse the messages by domain names, filter or sort by date, or author and recipient. And of course set aside duplicates and any messages that have already been reviewed. I mean, deduplication is something we in the industry do every day on millions and millions of electronic documents. Applying a file hash algorithm, the digital equivalent of the file’s fingerprint, to each message file and then comparing the hash values would reveal any duplicate files. Heck, near-deduplication technology could even show the slight differences between very similar files and group similar documents together or highlight the differences. Deduplication tools are particularly useful on email message files because these files typically contain a unique message ID that helps identify related messages.

The truth is that it’s fairly easy with the right software to get through 650,000 emails. Processing software extracts the header, message body, and fields of metadata (the To, From, Date, etc.) associated with each email message and within hours all that information can be loaded to a fully searchable database or document review platform. From there it take minutes to search, sort, and filter to narrow the files down to those that are relevant.

Not Without Limitations

Obviously, this is a high-level overview of the process, and to be sure, the volume of emails dictates how long it takes to process the files, but even 650,000 emails would be considered a small- to medium-sized project that could be handled in just a few days. Law firms, corporate legal departments and service providers in our industry do this every single day of the week. It should not come as a surprise that the government can do it too.

My guess would be that the laptop in this case contained a high volume of duplicate emails and email messages that that the government had already seen. So, what was reported as 650,000 emails was probably quickly reduced to a more manageable number. From there, the documents could easily be reviewed in a linear manner one by one, or additional machine learning, predictive analytic tools, or conceptual search tools could have been used to reduce the volume even further or focus on particular concepts of particular relevance.

But the short answer is, yes, it is possible to review 650,000 emails in eight days.

Who Put the “Artificial” in Artificial Intelligence

ai_1Okay, so there’s no way this does not come off as a complete rant. But bear with me; it’s just a question.

Why do we use the term artificial intelligence when referring to tools built on IBM’s Watson or other “cognitive” or machine learning technology? As far as I know, there’s nothing artificial about these tools. I mean, sure, they are artificial in that they are man-made, as opposed to a natural occurrence, but make no mistake, there’s nothing artificial about the math and science. Computing technology at its core involves ones and zeros, right? Well, maybe it’s not quite that simple, but the point is made.

Do you make artificial business decisions?

I’m no rocket scientist, but even to a person of limited legal, business, and/or technical ability it should seem odd to rely on something “artificial” to deliver sound, defensible results and outcomes that need to be, well, something more than artificial. I won’t speak for anyone else, but if I were making important legal or business decisions and the person or persons providing the technology that solves my problem told me the tool to be used was “artificial,” this quite frankly raises red flags for me.

To TAR or not to TAR

Recently, I’ve had several conversations about why the so-called TAR or predictive coding tools have not taken off in ways that many envisioned. TAR came in with a bang. Everyone rushed to check it out, and then . . . well, not much. I’d say the bottom fell out, but the truth is it never really filled the void, did it? Surveys reveal TAR-type products are employed in a very low percentage of cases. It simply has not seen broad adoption.

Well, I have a theory. And a prediction. First, TAR and predictive coding-type products did not take off because to some degree their usefulness in the legal business has been over-stated.  This does not mean they do not work. Others can debate the science of the technology. No, what is meant by usefulness is merely the frequency of application. TAR is not for every case. I guess theoretically it could be; but the point is that that is not the way it has been sold. Rather, it has been marketed as the next shiny thing to take the legal world by storm, promising to up-end the way lawyers work, and save millions of dollars and time in document review. And you know what? All of that is true –but only on a handful of cases.

Shiny objects are not sound processes

And this leads to my theory. I’m no marketing genius, but I am a pretty savvy consumer and I know that I consistently look for certain qualities when making purchasing decisions. I’m usually pretty good at looking past the marketing fluff and seeing a product for what it is. But I see a pattern in the legal technology industry. We keep coming up with the next shiny object and everyone, in what stock pickers call a “herd mentality,” migrates towards these objects, only to find out that the excitement is short-lived, that the product was not all that, or worse.

And now for the dangerous part –the prediction. It’s always dangerous to attempt to predict the future. But at this point I feel like the pattern may make it easier. Quite apart from the shiny object theory above, I predict that artificial intelligence in the legal business will underwhelm at best. I make this prediction because I’ve seen in the past –recall ECA, TAR and predictive coding—that smart, strong marketing pushed these tools to the forefront of everyone’s consciousness, but then they fade away. It’s like a new stock IPO –everyone is on-board day one, but when the novelty wears off someone is left holding the bag. Maybe this is why law firms are loathe to be first at things. But once a single high-profile firm weighs in, boy do the rest fall in line.

ai_0

This is all very cynical, I know, but to paraphrase something Pete Townsend once said: “It’s not that I am more cynical; it’s just that as I get older I have more evidence to support my cynicism.” As a long-time advocate for technology in the legal industry, I really hope that I am wrong about this.

 Back to the future (reality)

Putting aside all the shiny new objects and marketing campaigns, what is clear to me overall is the legal business needs change and in lieu of change it needs sound process. Yes, disruption is sometimes good and forges change. But it’s like the definition of insanity – if you keep doing the same things, how can you expect to achieve different results? What is needed in legal technology is a fresh, new attitude. Not one driven by profits, or that is designed to meet investor targets. No, I think it has to be an attitude of complete transparency, guided by principles of solving problems first. Every bit of business sense in my soul tells me that if you do good things, good things will come to you.

And in lieu of that, I always fall back on the notion that you need good processes. No amount of software in the world –no matter how “smart” it may be– is going to help fix a flawed process.  Before buying software in an attempt to make things more efficient, make sure you first identify the process that is in need of improvement.

The book is called Project Management in Electronic Discovery

Book Cover Title

I write in this book about the work that a project leader undertakes. But more importantly, I write here for lawyers, paralegals, students, and support staff at law firms and in corporate and government legal departments or at service providers. This book captures the principles of project management and the best practices of discovery in litigation, particularly as they relate to the management of large volumes of electronically stored information.

At the same time, it serves as a guide and reference for students of the law, paralegals, and attorneys, and illustrates how project management processes and technology may be used to provide efficient, client-oriented services and high-quality deliverables in a litigation support environment—at scope, on time, and within budget.

I am inspired here as much by the need for proper discourse on the use of project management in legal discovery projects as by the growing insistence that legal teams function more efficiently and cost effectively. That and the realization I came to recently when I was asked to help design and teach a project management class for students eager to break into the litigation support field. Try as we might, neither the university nor I could find an appropriate text for an electronic discovery project management class. An article I authored a few years ago that outlined some of the material presented here for a trade magazine served as a jumping off point for this book.

Another contributing factor has been my many working with lawyers and in law firms, most recently as the Director of Litigation Support Services at Stroock & Stroock & Lavan LLP. Putting in countless hours, making I imagine nearly every mistake possible, and observing a good deal of inefficiency, and now having designed and implemented litigation support and electronic discovery work-flows, I have some sense of what does and does not work. To be sure, not all project management principles strictly apply in the context of every legal discovery project. But many of the core principles can be applied and, in my experience working in the legal business, it has now become abundantly clear to me at least that using project management principles can and will serve law firms and their clients well.

Why write a book about project management and electronic discovery?

There has been considerable discussion regarding the applicability of project management in a legal setting. Driving the debate in part is the need of law firms and corporate legal departments to find efficiencies and reduce legal expenses. Law firms, which rely upon corporations for a slice of the $300 billion legal services market, have begun to listen. Most firms are adjusting hourly rates, staffing cases leanly, or have entered into unique and tailored billing agreements. Fewer firms, however, have dramatically altered the ways in which lawyers work and fewer still have adopted project management principles into their business model. This book addresses how project management may be used in a legal setting to make one aspect of legal work more efficient—the discovery process.
A distinction needs to be made between applying project management principles to the entire operation of a law firm and the application of project management to discreet aspects of legal practice. It is of course possible to apply project management methodologies to the operation of a law firm, from business development to matter management and throughout the individual cases, transactions, and subordinate tasks that make up the core of attorney practice. Indeed, a few firms focus on just that. The emphasis in my book, though, is on the application of project management principles in the context of providing litigation support services and the integration of project-oriented processes into managing legal discovery projects and, more specifically, electronic discovery projects.
There is much work to do in this area. In the United States, which is 80% of the global market, the electronic discovery market is growing incredibly fast. The market worth of the global electronic discovery industry has doubled since 2010 and is projected to quadruple by 2020. Driven by the massive growth of electronically stored information and the need to manage that information for civil litigation, the e-discovery market, including services and software, grew to over $7 billion in 2015. A recent report by Transparency Market Research projects 16% compound annual growth rate for services and software through 2022, increasing the market to more than $21 billion. A 2015 report by Gartner similarly projects double-digit year-over-year U.S. growth in e-discovery software.
However, due in part to the recession and the slower than expected economic recovery, the legal services industry is undergoing considerable change. Law firms have folded, others have merged, and even the best firms have reduced personnel by 10% to 30% in the past six years. Corporate legal department budgets are shrinking, and they face pressure to reduce legal expenses. One need not have an MBA or a law degree to conclude that the current business model at many law firms may not be sustainable in the long term. To survive, lawyers and legal support staff in the U.S. and abroad—those at larger law firms in particular—need to change the way they practice and deliver legal services. Applying project management methodologies to the practice of law is one tool that will differentiate great firms from good firms and provide the efficiency and sustainability needed in a legal market that is very different today from just a few years ago. Additionally, project management brings structure and common business sense to law firms, which traditionally are not run like a business.
By far, the most costly and time consuming aspect of litigation is the discovery process. Electronic discovery is the prime suspect in what is a disproportionate increase in discovery costs relative to the overall expense of litigation. It used to be that junior lawyers gathered in rooms filled with boxes to review documents. Technology has changed this. Discovery is no longer about boxes of paper; instead, it is about terabytes of data.  Information is now everywhere. Email and text messaging are ubiquitous, and social media, smart phones, and the Internet have invaded our lives. All this data is discoverable in litigation. Discovery is not more costly because of inflation or because attorney hourly rates have risen. In fact, the costs associated with discovery, particularly with electronic discovery, have gone down considerably. Discovery costs more today because there is so much more information and costs are driven almost entirely by the volume of data. These things—the volume of data, the time and cost it takes to manage this information—understandably have corporations, law firms, and the government concerned.
In a world overrun by electronically stored information, it should not surprise anyone that new processes, new technologies, and a new breed of legal support personnel have emerged. In the mid-1990s, as computers began to arrive at every employees’ desk, it became abundantly clear that email and electronic documents were a new source of discovery material. Still, it took years before for the legal community prepared rules that even acknowledged the discovery of electronic documents. Today, almost all documents are created on computers and they are stored not in file cabinets or boxes, but on optical disk and in databases. Analytics software, long used in other industries, is now used to parse through vast amounts of data and to find documents that make or break a case. The litigation support industry and the process of electronic discovery were born out of the need to integrate technology into the legal process. People who work in the industry provide technical support, software, project management, and consulting services to law firms, corporate legal departments, and the government. A completely new body of case law and rules governing the proper preservation, collection, review, and production of electronic documents has also emerged. This once nascent industry is beginning to mature and stabilize, and it seeks standardization and increased efficiency.
This is the backdrop for a book that introduces new and better ways to manage electronic discovery projects.