This partly technical article argues that question-answering systems built on top of improved ‘deep’ versions of traditional NLP-algorithms will have a profound impact on the legal sector in the years to come. The reasons for this are twofold. First, deep neural networks will vastly improve the quality of predictions for various NLP tasks relevant to uses cases in the legal profession. Second, we will be able to build better abstractions that make complex system a delight to use even for nontechnical users.
In recent years, we have seen an explosion of the use of ‘deep learning’ in various domains, ranging from image recognition over natural language processing (NLP) to game playing. Old ideas from artificial neural networks combined with modern parallel computing infrastructure, the availability of massive amounts of data and open source tools such as TensorFlow or Theano enable us to efficiently build machine learning models that perform surprisingly well even with no or very limited nitty-gritty feature engineering or domain knowledge. Due to improved tooling, setting up and training a deep neural network doesn’t require spending nights in the office taking symbolic derivatives for each layer only to figure out later that the algorithm gets stuck in local optima all the time for mysterious reasons – an experience graduate students in statistics and machine learning may be able to relate to.
In NLP, deep neutral networks recently blew traditional methods out of the water for various NLP-tasks, such as machine translation. If you recently used Google Translate for translations between the the major language pairs, you may have been surprised about the improved translation quality (the New York Times devoted an article to this). The fact that the challenge of creating representations of real world entities such as text or pictures useful for a machine learning algorithm can in part be done by the computer itself takes a lot of feature-engineering headaches out of machine learning. There is a great article by Christopher Olah on this topic in the context of NLP.
Partly thanks to the advances in NLP, the “great A. I. awakening” in the legal domain will play out over the next years. But which NLP tasks and technologies will have the biggest impact and how exactly will deep learning change workflows of law professionals? The goal of this article is to give some insights on these questions by discussing various NLP tasks and their use cases in the legal domain.
Where are my employment contracts? – Document classification to the rescue
Document classification – the problem of automatically assigning a document to one or more (pre-defined) classes – is a rather well researched topic in machine learning and information retrieval. F1 scores of almost 100% – depending on the (size of the) corpus – have been achieved on the main benchmark (Reuters-21578 collection) for this task using traditional classification methods (SVM, KNN). A recent paper by Lai et. al. shows that document classification with recurrent convolutional neural networks achieves similar performances even without human-designed features.
One of the use cases in the legal sector is document search and filtering. Machine learning algorithms can easily consume vast document repositories and classify them such that users can easily search and filter them based on various classifications. Organizing documents in deep folder structures will not be necessary anymore, since search will become more useful thanks to the power of automatic document classification.
What are the names of my counterparties? – Named entity recognition
Named entity recognition (NER) seeks to automatically locate and classify named entities into pre-defined categories such as persons, locations, times, quantitative values and the like. For example, the picture highlights all named corporate entities identified by an algorithm. Most current tools, such as the Stanford Named Entity Recognizer, rely on conditional random fields, a sequence modeling technique that is known to work very well for NER. Recently, convolutional neural networks achieved similar performances in some experiments (see for example this article by Eric Yuan).
A great application of NER in the legal domain is to answer specific questions about large sets of legal documents. Here are some examples:
– How many service contracts does company X have with company Y?
– What is the total amount of rent payments company X receives in 2017 based on all rental contracts with other parties?
Apart from that, NER can help in creating contract templates from actual contracts by eliminating recognized named entities and in harvesting and visualizing large amounts of quantitative data hidden in legal documents.
Who is “the seller”? – Coreference resolution
Coreference resolution is the problem of finding all expressions in text that refer to the same entity. The picture shows a simple example where an algorithm infers that “BMW” is equivalent to “BMW Group” (and not equivalent to “BMW AG”) in a given document. Coreference resolution is a well studied problem. Nevertheless, algorithms currently have a rather low accuracy of around 75% and the tradeoff between precision and recall is rather delicate. Recent results using neural networks point to models that may achieve better performance.
When reading legal documents, it is not always clear to which entity a certain word or set of words refers to. Therefore, an obvious application of coreference resolution would be to identify all references to the same entity in a legal document. This references could then be resolved to the actual entity, such that it is always clear what exactly is meant with “the seller”.
Where is the safeguard provision? – Topic classification
Topic classification is the task of assigning topics to predefined parts of a text. Technically, topic classification is very similar to document classification, but usually deals with shorter texts or even with paragraphs only. Usually, the shorter the text, the harder it is to extract signals that have enough power to distinguish topic labels. The picture shows a paragraph an algorithm has classified as a precedence clause. A paper by Nho & Ng compares different methods for paragraph topic classification.
In the legal domain, one of the most important use cases is to automatically identify certain sets of rules in legal documents, such as safeguarding clauses in employment contracts. This also allows professionals to compare this rules across contracts and see how the details of rules differ across contracts.
Why can’t I ask in natural language right away? – Question answering
I introduced the above sections with questions that can be answered by performing certain NLP-tasks. However, answering these questions would still require a user to perform some nontrivial analysis with a computer. By building question answering systems on top of the above algorithms, users will be able to pose the above questions directly in natural language. It will then be very easy for legal professionals to use the power of A.I. in their daily work. Moreover, contrary to what currently seems to be conventional wisdom, legal professionals do not need to become experts in computer science. By getting the abstractions right, complexity can be hidden from the user and professionals may be able to interact with computers in a similar way they would with humans.