# Discovering Explanatory Sentences in Legal Case Decisions Using Pre-trained Language Models

arXiv: https://arxiv.org/abs/2112.07165 · HTML: https://mlsanity.com/paper/2112.07165/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 40 references that are in the map.
- First posted on arXiv 14 Dec 2021.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [A Multiscale Visualization of Attention in the Transformer Model](https://mlsanity.com/paper/1906.05714/) (2019)
- [Pretrained Transformers for Text Ranking: BERT and Beyond](https://mlsanity.com/paper/2010.06467/) (2020)
- [Neural Legal Judgment Prediction in English](https://mlsanity.com/paper/1906.02059/) (2019)
- [Multi-Stage Document Ranking with BERT](https://mlsanity.com/paper/1910.14424/) (2019)
- [Simple Applications of BERT for Ad Hoc Document Retrieval](https://mlsanity.com/paper/1903.10972/) (2019)
- [Legal Area Classification: A Comparative Study of Text Classifiers on Singapore Supreme Court Judgments](https://mlsanity.com/paper/1904.06470/) (2019)
- [Legal Search in Case Law and Statute Law](https://mlsanity.com/paper/2108.10127/) (2021)

## Built on by (most cited first, 9 of 9)

- [Can GPT-3 Perform Statutory Reasoning?](https://mlsanity.com/paper/2302.06100/) (2023)
- [Explaining Legal Concepts with Augmented Large Language Models (GPT-4)](https://mlsanity.com/paper/2306.09525/) (2023)
- [Can GPT-4 Support Analysis of Textual Data in Tasks Requiring Highly Specialized Domain Expertise?](https://mlsanity.com/paper/2306.13906/) (2023)
- [Bringing order into the realm of Transformer-based language models for artificial intelligence and law](https://mlsanity.com/paper/2308.05502/) (2023)
- [Prototype-Based Interpretability for Legal Citation Prediction](https://mlsanity.com/paper/2305.16490/) (2023)
- [An Intent Taxonomy of Legal Case Retrieval](https://mlsanity.com/paper/2307.13298/) (2023)
- [Automating Legal Interpretation with LLMs: Retrieval, Generation, and Evaluation](https://mlsanity.com/paper/2501.01743/) (2025)
- [Can Generative Pre-trained Transformers (GPT) Pass Assessments in Higher Education Programming Courses?](https://mlsanity.com/paper/2303.09325/) (2023)
- [Are manual annotations necessary for statutory interpretations retrieval?](https://mlsanity.com/paper/2506.13965/) (2025)

## Abstract (arXiv)

Legal texts routinely use concepts that are difficult to understand. Lawyers elaborate on the meaning of such concepts by, among other things, carefully investigating how have they been used in past. Finding text snippets that mention a particular concept in a useful way is tedious, time-consuming, and, hence, expensive. We assembled a data set of 26,959 sentences, coming from legal case decisions, and labeled them in terms of their usefulness for explaining selected legal concepts. Using the dataset we study the effectiveness of transformer-based models pre-trained on large language corpora to detect which of the sentences are useful. In light of models' predictions, we analyze various linguistic properties of the explanatory sentences as well as their relationship to the legal concept that needs to be explained. We show that the transformer-based models are capable of learning surprisingly sophisticated features and outperform the prior approaches to the task.