Alternative Facts and Artificial Intelligence

A few months back, a post here began by asking “Who Put the ‘Artificial’ in Artificial Intelligence?” At the time, I was sort of complaining about low adoption rates for technology tools in legal. I suggested that good marketing and “shiny object syndrome” are responsible for the rise of new technologies in legal. In the end, I question whether it’s appropriate to describe technology tools in the legal business as “artificial intelligence” tools.

Generally, I hardly think of myself as an “I told you so” type of person. If I’m being honest, though, I do think I told you so more than I articulate I told you so. (I like to think I’ve learned something tactful over the years). But an interview on got me to thinking.

IBM Logo Artificial IntelligenceFortune spoke to David Kenny, leader of the Watson Group at IBM. I thought while reading the interview that if anyone has a good grasp of artificial intelligence it’s IBM, right? Well, I have news: the word “artificial” really has no place in the technology tools that are in use in the legal industry—at least not the ones in use in e-discovery.

It’s Elementary My Dear Watson

To distinguish between the many AI terms out there, David Kenny said,

“Deep learning uses more advanced things like convolutional neural networks, which basically means you can look at things more deeply in more layers. Machine learning could work, for example, when it came to reading text. Deep learning was needed when we wanted to read an X-ray. And all of that has led to this concept of artificial intelligence—though at IBM, we tend to say, in many cases, that it’s not artificial as much as it’s augmented. So, it’s a system between machine computing and humans interpreting, and we call those machine-human interactions cognitive systems.”

“As for what we would call unsupervised learning—which is to say, we’re not training it to process but it’s beginning to learn on its own—that is moving more in the direction of what some consider true artificial intelligence, or even AGI: artificial general intelligence. I would say we’re at the early stages of that.”

Explaining how Watson is able to ingest the text of 26 million medical and scientific articles to help doctors find clinical trials for cancer patients, Mr. Kenny said:

“. . . it starts with knowledge extraction: reading documents, finding common phrases, associating those together. It does the same with paragraphs. Then it has to get corrected. The human annotation is critical here: Out of the gate there’s no way that I would trust the system to do unsupervised learning and just find the patterns on its own. You literally tell Watson, “Yes, that meant this, yes those go together. Yes, you have that right, or, no, you don’t.”

“And when you tell the system the “no’s,” it re-weights its algorithms until it gets to a point where it would have produced the correct answer. And it gets better over time.”

True Artificial Intelligence: Who’s Supervising Who

There you have it, folks. That is machine learning, hands down. And it’s precisely how the technology applies in e-discovery. It’s not a stretch, then, to say that what we are currently doing in e-discovery is not artificial intelligence. Additionally, what many now refer to as artificial intelligence is actually machine learning. Or, for the marketing folks out there, augmented intelligence (it’s still “AI” but not quite as sexy, right?)

To call it anything else, I think, is irresponsible and misleading. Lest we forget –and unless I missed something—the TAR, predictive and analytic programs in use in legal are algorithmic tools that require some level of human input. That’s not AI, as Kenny points out, because true AI is not supervised.

Why Does This Matter?

It matters because I think it’s not accurate. The reason it is “augmented” and not “artificial” in its application in the legal business is fairly obvious. But in addition, as far as I know, there are no lawyers –none who want to keep their licenses anyway—who are making important decisions, giving advice to clients, or reviewing and producing documents based on true artificial intelligence. No one plugs documents into a computer and spits them out without lawyers being involved (at least I hope not). Seems to me that there’s always a human to some degree augmenting the process.

Now, I’m not a scientist or a linguist. Heck, I barely qualify as a writer. It’s troubling, though, that in an industry in which facts and a search for the truth are paramount too much time and money goes to promoting products and ideas that are, how should I say, “artificial.”

I think it was the first celebrity law enforcement officer, Joe Friday, who said “Just the facts Ma’am.” So, let’s just stick to the facts in e-discovery and try not to promote tools that promise the stars and regrettably fall short. Ultimately, I think doing so will help foster adoption of technology in the legal industry.

I’d love to hear your thoughts on the issue, especially those of the scientists out there.

Can 650,000 Emails Be Reviewed In Eight Days?

workflowheaderIn the email debacle that engulfed the presidential election this past week, the question arose whether the government could possibly review 650,000 emails in eight days. With today’s technology, not only is it possible, but those who work in the litigation/practice support business know full well how it might be done. What surprises me I guess is that more people don’t realize that it’s possible.

My friend Eric Mandel recently wrote on LinkedIn about what could be learned in just 72 hours about the new emails. And I commented that I don’t understand how we in the industry could understand this, but the government does not. Well, apologies to the government – apparently they were doing (had to be doing) what we suspected all along –using technology to churn through the emails to determine whether there was anything of substance there.

We Have the Technology

For the uninitiated, using technology available today –technology, by the way that has been available for a long time— the government could not only parse through the email on that laptop in no time at all, they could also gain some real insight into the substance of the documents. To conclude otherwise, would be to deny science (and that’s not something politicians do, is it?).

How did they do this? As Eric pointed out, the initial question is whether this new tranche of emails contains anything new. One might start by doing some basic filtering and culling. Eliminate junk email, parse the messages by domain names, filter or sort by date, or author and recipient. And of course set aside duplicates and any messages that have already been reviewed. I mean, deduplication is something we in the industry do every day on millions and millions of electronic documents. Applying a file hash algorithm, the digital equivalent of the file’s fingerprint, to each message file and then comparing the hash values would reveal any duplicate files. Heck, near-deduplication technology could even show the slight differences between very similar files and group similar documents together or highlight the differences. Deduplication tools are particularly useful on email message files because these files typically contain a unique message ID that helps identify related messages.

The truth is that it’s fairly easy with the right software to get through 650,000 emails. Processing software extracts the header, message body, and fields of metadata (the To, From, Date, etc.) associated with each email message and within hours all that information can be loaded to a fully searchable database or document review platform. From there it take minutes to search, sort, and filter to narrow the files down to those that are relevant.

Not Without Limitations

Obviously, this is a high-level overview of the process, and to be sure, the volume of emails dictates how long it takes to process the files, but even 650,000 emails would be considered a small- to medium-sized project that could be handled in just a few days. Law firms, corporate legal departments and service providers in our industry do this every single day of the week. It should not come as a surprise that the government can do it too.

My guess would be that the laptop in this case contained a high volume of duplicate emails and email messages that that the government had already seen. So, what was reported as 650,000 emails was probably quickly reduced to a more manageable number. From there, the documents could easily be reviewed in a linear manner one by one, or additional machine learning, predictive analytic tools, or conceptual search tools could have been used to reduce the volume even further or focus on particular concepts of particular relevance.

But the short answer is, yes, it is possible to review 650,000 emails in eight days.

Who Put the “Artificial” in Artificial Intelligence

ai_1Okay, so there’s no way this does not come off as a complete rant. But bear with me; it’s just a question.

Why do we use the term artificial intelligence when referring to tools built on IBM’s Watson or other “cognitive” or machine learning technology? As far as I know, there’s nothing artificial about these tools. I mean, sure, they are artificial in that they are man-made, as opposed to a natural occurrence, but make no mistake, there’s nothing artificial about the math and science. Computing technology at its core involves ones and zeros, right? Well, maybe it’s not quite that simple, but the point is made.

Do you make artificial business decisions?

I’m no rocket scientist, but even to a person of limited legal, business, and/or technical ability it should seem odd to rely on something “artificial” to deliver sound, defensible results and outcomes that need to be, well, something more than artificial. I won’t speak for anyone else, but if I were making important legal or business decisions and the person or persons providing the technology that solves my problem told me the tool to be used was “artificial,” this quite frankly raises red flags for me.

To TAR or not to TAR

Recently, I’ve had several conversations about why the so-called TAR or predictive coding tools have not taken off in ways that many envisioned. TAR came in with a bang. Everyone rushed to check it out, and then . . . well, not much. I’d say the bottom fell out, but the truth is it never really filled the void, did it? Surveys reveal TAR-type products are employed in a very low percentage of cases. It simply has not seen broad adoption.

Well, I have a theory. And a prediction. First, TAR and predictive coding-type products did not take off because to some degree their usefulness in the legal business has been over-stated.  This does not mean they do not work. Others can debate the science of the technology. No, what is meant by usefulness is merely the frequency of application. TAR is not for every case. I guess theoretically it could be; but the point is that that is not the way it has been sold. Rather, it has been marketed as the next shiny thing to take the legal world by storm, promising to up-end the way lawyers work, and save millions of dollars and time in document review. And you know what? All of that is true –but only on a handful of cases.

Shiny objects are not sound processes

And this leads to my theory. I’m no marketing genius, but I am a pretty savvy consumer and I know that I consistently look for certain qualities when making purchasing decisions. I’m usually pretty good at looking past the marketing fluff and seeing a product for what it is. But I see a pattern in the legal technology industry. We keep coming up with the next shiny object and everyone, in what stock pickers call a “herd mentality,” migrates towards these objects, only to find out that the excitement is short-lived, that the product was not all that, or worse.

And now for the dangerous part –the prediction. It’s always dangerous to attempt to predict the future. But at this point I feel like the pattern may make it easier. Quite apart from the shiny object theory above, I predict that artificial intelligence in the legal business will underwhelm at best. I make this prediction because I’ve seen in the past –recall ECA, TAR and predictive coding—that smart, strong marketing pushed these tools to the forefront of everyone’s consciousness, but then they fade away. It’s like a new stock IPO –everyone is on-board day one, but when the novelty wears off someone is left holding the bag. Maybe this is why law firms are loathe to be first at things. But once a single high-profile firm weighs in, boy do the rest fall in line.


This is all very cynical, I know, but to paraphrase something Pete Townsend once said: “It’s not that I am more cynical; it’s just that as I get older I have more evidence to support my cynicism.” As a long-time advocate for technology in the legal industry, I really hope that I am wrong about this.

 Back to the future (reality)

Putting aside all the shiny new objects and marketing campaigns, what is clear to me overall is the legal business needs change and in lieu of change it needs sound process. Yes, disruption is sometimes good and forges change. But it’s like the definition of insanity – if you keep doing the same things, how can you expect to achieve different results? What is needed in legal technology is a fresh, new attitude. Not one driven by profits, or that is designed to meet investor targets. No, I think it has to be an attitude of complete transparency, guided by principles of solving problems first. Every bit of business sense in my soul tells me that if you do good things, good things will come to you.

And in lieu of that, I always fall back on the notion that you need good processes. No amount of software in the world –no matter how “smart” it may be– is going to help fix a flawed process.  Before buying software in an attempt to make things more efficient, make sure you first identify the process that is in need of improvement.

2016 ACEDS Conference & Exhibition

I attended the 2016 ACEDS Conference & Exhibition of the Association of Certified E-Discovery Specialists at the Grand Hyatt in New York City last week. Quite a show as usual. The leadership and management at ACEDS are a truly dedicated group of individuals committed to education and the growth and development of the e-discovery and litigation support industry. You can check out their content on the web here.

This year featured top talent from all corners of the e-discovery and litigation support industry. Now led by industry veteran Mary Mack, ACEDS is poised to do great things. But more than anything, I think their commitment to education is unparalleled. Day one included several sessions devoted to privacy and cyber security, the new hot topics folks are talking about. Sessions on forensics, hiring and analytics were presented as well. As were several sessions on project management (my favorite topic), including a session I moderated entitled “E-Discovery Project Management: Ask Forgiveness Not Permission.” The crux of what we presented is that while an organizational project management initiative may involve high-level buy in and loads of resources, you don’t need permission to undertake project management protocols within your discreet department. In other words, even tough your firm or organization may not have a formal PMO or the resources to launch one, there is nothing that prevents you from using project management methodologies and protocols to make the work of your department efficient. You don’t need permission to save money. To mitigate risk. To be more efficient. It was a great session. I’m preparing a separate post to illustrate how project management methodologies may be used specifically to manage your litigation support department. And keep an eye out for recordings of the session on the ACEDS site in the future.

The second day of the conference featured sessions on e-discovery education, ethical standards, and how to interview for a job in the industry. A great session with panelists from the judiciary addressed questions revolving around tracking terrorism and the Apple iPhone debate. There were also panel discussions on topics like bringing litigation support in-house, the power of affiliation with associations (a panel moderated by my good friend Maribel Rivera), avoiding sanctions and using six sigma for process improvement. Craig Ball’s presentation on the future of e-discovery was impressive too. And there was content for lawyers too, proving that ACEDS is not just a litigation/practice support organization.

And then came the awards luncheon. I was genuinely surprised, deeply honored and humbled to receive the ACEDS Person of the Year award. It’s great to be recognized by your peers. Thank you to those who nominated and voted for me. Many other people were recognized as well, including my friend Bill Hamilton, who received the Executive Director’s Award, and Craig Ball, who was awarded a Lifetime Achievement Award. The local NY Chapter of ACEDS was also awarded ACEDS Chapter of the Year. And congratulations to all the other award recipients (a list of which you’ll find here).

All in all, the conference was a huge success. As ACEDS continues to build it’s brand, it will become apparent that the organization is in this for the long haul. Their certification exam is rigorous and challenging and establishes a baseline for e-discovery competence in the industry. Their educational content is top-notch and spans a broad spectrum. And their people, led by Mary, Susan, James, and Al Lindsey, the chair of the advisory board, really have a finger or two on the pulse of what the industry needs.

4th Annual University of Florida EDRM E-Discovery Conference

Just back from Gainesville and the annual University of Florida EDRM E-Discovery Conference and once again congratulations to Bill Hamilton, George Socha, and a fantastic conference staff for putting together a top-quality event full of e-discovery education, camaraderie and conversation. What’s cool about the UF-EDRM conference is that you can attend in person or watch and listen online since the entire conference is streamed live over the Internet. I’m honored to be part of this conference again this year. Look for some pics in the Photo Gallery.

I spoke on a great panel related to the collection of ESI with fellow panelists Bob Rudnicki, Megan Kelberman and Jon Rowe. What is different about the annual UF-EDRM conference is it’s unique approach to how sessions are presented. Each panel consists of speakers, usually experts on the topic, who identify and discuss an issue or problem in the industry, and then there is typically a software provider who demonstrates how the software may help solve the problem. In this way it is unlike other conferences.

In our session, Bob Rudnicki presented the problem or “headache” of a putative class action lawsuit with the potential need to collect ESI from thousands of brokers around the nation. Megan and I (as the lawyer and litigation support professional, respectively) provided some “aspirin” and commentary on how Bob might go about solving this problem. And then Jon demonstrated his easy to use and defensible collection software. We also had great questions from the audience.

2016 UDEDRM Panel

The bottom line with collections, folks, is to focus on only the sources of ESI that will yield potentially relevant materials, use defensible tools that write-protect the data when it is copied, and document the process using a collection log and appropriate chain of custody.

Why you should read this book . . .

If the focus of your work or a goal in your career is learning the principles of project management, how they apply in a legal support setting, and how to use these principles to improve litigation support and electronic discovery deliverables in the legal industry, this book is the most comprehensive exposition on these subjects to date. Not only will it provide an understanding of the basic principles of traditional project management, this book also outlines the best practices in a relatively young industry in search of standardization. These two things make this book incredibly valuable to the novice. In addition, for the experienced practitioner and the journeymen in the industry, this book will provide a useful reference for years to come

The book is called Project Management in Electronic Discovery

Book Cover Title

I write in this book about the work that a project leader undertakes. But more importantly, I write here for lawyers, paralegals, students, and support staff at law firms and in corporate and government legal departments or at service providers. This book captures the principles of project management and the best practices of discovery in litigation, particularly as they relate to the management of large volumes of electronically stored information.

At the same time, it serves as a guide and reference for students of the law, paralegals, and attorneys, and illustrates how project management processes and technology may be used to provide efficient, client-oriented services and high-quality deliverables in a litigation support environment—at scope, on time, and within budget.

I am inspired here as much by the need for proper discourse on the use of project management in legal discovery projects as by the growing insistence that legal teams function more efficiently and cost effectively. That and the realization I came to recently when I was asked to help design and teach a project management class for students eager to break into the litigation support field. Try as we might, neither the university nor I could find an appropriate text for an electronic discovery project management class. An article I authored a few years ago that outlined some of the material presented here for a trade magazine served as a jumping off point for this book.

Another contributing factor has been my many working with lawyers and in law firms, most recently as the Director of Litigation Support Services at Stroock & Stroock & Lavan LLP. Putting in countless hours, making I imagine nearly every mistake possible, and observing a good deal of inefficiency, and now having designed and implemented litigation support and electronic discovery work-flows, I have some sense of what does and does not work. To be sure, not all project management principles strictly apply in the context of every legal discovery project. But many of the core principles can be applied and, in my experience working in the legal business, it has now become abundantly clear to me at least that using project management principles can and will serve law firms and their clients well.

Why write a book about project management and electronic discovery?

There has been considerable discussion regarding the applicability of project management in a legal setting. Driving the debate in part is the need of law firms and corporate legal departments to find efficiencies and reduce legal expenses. Law firms, which rely upon corporations for a slice of the $300 billion legal services market, have begun to listen. Most firms are adjusting hourly rates, staffing cases leanly, or have entered into unique and tailored billing agreements. Fewer firms, however, have dramatically altered the ways in which lawyers work and fewer still have adopted project management principles into their business model. This book addresses how project management may be used in a legal setting to make one aspect of legal work more efficient—the discovery process.
A distinction needs to be made between applying project management principles to the entire operation of a law firm and the application of project management to discreet aspects of legal practice. It is of course possible to apply project management methodologies to the operation of a law firm, from business development to matter management and throughout the individual cases, transactions, and subordinate tasks that make up the core of attorney practice. Indeed, a few firms focus on just that. The emphasis in my book, though, is on the application of project management principles in the context of providing litigation support services and the integration of project-oriented processes into managing legal discovery projects and, more specifically, electronic discovery projects.
There is much work to do in this area. In the United States, which is 80% of the global market, the electronic discovery market is growing incredibly fast. The market worth of the global electronic discovery industry has doubled since 2010 and is projected to quadruple by 2020. Driven by the massive growth of electronically stored information and the need to manage that information for civil litigation, the e-discovery market, including services and software, grew to over $7 billion in 2015. A recent report by Transparency Market Research projects 16% compound annual growth rate for services and software through 2022, increasing the market to more than $21 billion. A 2015 report by Gartner similarly projects double-digit year-over-year U.S. growth in e-discovery software.
However, due in part to the recession and the slower than expected economic recovery, the legal services industry is undergoing considerable change. Law firms have folded, others have merged, and even the best firms have reduced personnel by 10% to 30% in the past six years. Corporate legal department budgets are shrinking, and they face pressure to reduce legal expenses. One need not have an MBA or a law degree to conclude that the current business model at many law firms may not be sustainable in the long term. To survive, lawyers and legal support staff in the U.S. and abroad—those at larger law firms in particular—need to change the way they practice and deliver legal services. Applying project management methodologies to the practice of law is one tool that will differentiate great firms from good firms and provide the efficiency and sustainability needed in a legal market that is very different today from just a few years ago. Additionally, project management brings structure and common business sense to law firms, which traditionally are not run like a business.
By far, the most costly and time consuming aspect of litigation is the discovery process. Electronic discovery is the prime suspect in what is a disproportionate increase in discovery costs relative to the overall expense of litigation. It used to be that junior lawyers gathered in rooms filled with boxes to review documents. Technology has changed this. Discovery is no longer about boxes of paper; instead, it is about terabytes of data.  Information is now everywhere. Email and text messaging are ubiquitous, and social media, smart phones, and the Internet have invaded our lives. All this data is discoverable in litigation. Discovery is not more costly because of inflation or because attorney hourly rates have risen. In fact, the costs associated with discovery, particularly with electronic discovery, have gone down considerably. Discovery costs more today because there is so much more information and costs are driven almost entirely by the volume of data. These things—the volume of data, the time and cost it takes to manage this information—understandably have corporations, law firms, and the government concerned.
In a world overrun by electronically stored information, it should not surprise anyone that new processes, new technologies, and a new breed of legal support personnel have emerged. In the mid-1990s, as computers began to arrive at every employees’ desk, it became abundantly clear that email and electronic documents were a new source of discovery material. Still, it took years before for the legal community prepared rules that even acknowledged the discovery of electronic documents. Today, almost all documents are created on computers and they are stored not in file cabinets or boxes, but on optical disk and in databases. Analytics software, long used in other industries, is now used to parse through vast amounts of data and to find documents that make or break a case. The litigation support industry and the process of electronic discovery were born out of the need to integrate technology into the legal process. People who work in the industry provide technical support, software, project management, and consulting services to law firms, corporate legal departments, and the government. A completely new body of case law and rules governing the proper preservation, collection, review, and production of electronic documents has also emerged. This once nascent industry is beginning to mature and stabilize, and it seeks standardization and increased efficiency.
This is the backdrop for a book that introduces new and better ways to manage electronic discovery projects.