# Explaining Legal Concepts with Augmented Large Language Models (GPT-4)

arXiv: https://arxiv.org/abs/2306.09525 · HTML: https://mlsanity.com/paper/2306.09525/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 38 references that are in the map.
- First posted on arXiv 15 Jun 2023.

## Builds on (most cited first, 17 of 17)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [Reading Wikipedia to Answer Open-Domain Questions](https://mlsanity.com/paper/1704.00051/) (2017)
- [Augmented Language Models: a Survey](https://mlsanity.com/paper/2302.07842/) (2023)
- [BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage](https://mlsanity.com/paper/2208.03188/) (2022)
- [Can GPT-3 Perform Statutory Reasoning?](https://mlsanity.com/paper/2302.06100/) (2023)
- [Legal Prompting: Teaching a Language Model to Think Like a Lawyer](https://mlsanity.com/paper/2212.01326/) (2022)
- [One Question Answering Model for Many Languages with Cross-lingual Dense Passage Retrieval](https://mlsanity.com/paper/2107.11976/) (2021)
- [Blind Judgement: Agent-Based Supreme Court Modelling With GPT](https://mlsanity.com/paper/2301.05327/) (2023)
- [Unlocking Practical Applications in Legal Domain: Evaluation of GPT for Zero-Shot Semantic Annotation of Legal Texts](https://mlsanity.com/paper/2305.04417/) (2023)
- [How well do SOTA legal reasoning models support abductive reasoning?](https://mlsanity.com/paper/2304.06912/) (2023)
- [Discovering Explanatory Sentences in Legal Case Decisions Using Pre-trained Language Models](https://mlsanity.com/paper/2112.07165/) (2021)

## Built on by (most cited first, 30 of 40)

- [A Comprehensive Overview of Large Language Models](https://mlsanity.com/paper/2307.06435/) (2023)
- [Continual Learning of Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2404.16789/) (2024)
- [Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity](https://mlsanity.com/paper/2310.07521/) (2023)
- [Evaluating Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2310.19736/) (2023)
- [Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models](https://mlsanity.com/paper/2401.01301/) (2024)
- [SaulLM-7B: A pioneering Large Language Model for Law](https://mlsanity.com/paper/2403.03883/) (2024)
- [A Comprehensive Survey of Hallucination in Large Language, Image, Video and Audio Foundation Models](https://mlsanity.com/paper/2405.09589/) (2024)
- [LEXam: Benchmarking Legal Reasoning on 340 Law Exams](https://mlsanity.com/paper/2505.12864/) (2025)
- [Natural Language Processing for the Legal Domain: A Survey of Tasks, Datasets, Models, and Challenges](https://mlsanity.com/paper/2410.21306/) (2024)
- [Can GPT-4 Support Analysis of Textual Data in Tasks Requiring Highly Specialized Domain Expertise?](https://mlsanity.com/paper/2306.13906/) (2023)
- [From Lived Experience to Insight: Unpacking the Psychological Risks of Using AI Conversational Agents](https://mlsanity.com/paper/2412.07951/) (2024)
- [Leveraging Large Language Models for Learning Complex Legal Concepts through Storytelling](https://mlsanity.com/paper/2402.17019/) (2024)
- [LLM for SoC Security: A Paradigm Shift](https://mlsanity.com/paper/2310.06046/) (2023)
- [From Text to Structure: Using Large Language Models to Support the Development of Legal Expert Systems](https://mlsanity.com/paper/2311.04911/) (2023)
- [A Comprehensive Survey on Evaluating Large Language Model Applications in the Medical Industry](https://mlsanity.com/paper/2404.15777/) (2024)
- [CitaLaw: Enhancing LLM with Citations in Legal Domain](https://mlsanity.com/paper/2412.14556/) (2024)
- [ERAGent: Enhancing Retrieval-Augmented Language Models with Improved Accuracy, Efficiency, and Personalization](https://mlsanity.com/paper/2405.06683/) (2024)
- [Robots in the Middle: Evaluating LLMs in Dispute Resolution](https://mlsanity.com/paper/2410.07053/) (2024)
- [LegalLens: Leveraging LLMs for Legal Violation Identification in Unstructured Text](https://mlsanity.com/paper/2402.04335/) (2024)
- [BenchmarkCards: Standardized Documentation for Large Language Model Benchmarks](https://mlsanity.com/paper/2410.12974/) (2024)
- [Evaluating Test-Time Scaling LLMs for Legal Reasoning: OpenAI o1, DeepSeek-R1, and Beyond](https://mlsanity.com/paper/2503.16040/) (2025)
- [LLMediator: GPT-4 Assisted Online Dispute Resolution](https://mlsanity.com/paper/2307.16732/) (2023)
- [Exploring the Nexus of Large Language Models and Legal Systems: A Short Survey](https://mlsanity.com/paper/2404.00990/) (2024)
- [Getting in the Door: Streamlining Intake in Civil Legal Services with Large Language Models](https://mlsanity.com/paper/2410.03762/) (2024)
- [From Standalone LLMs to Integrated Intelligence: A Survey of Compound Al Systems](https://mlsanity.com/paper/2506.04565/) (2025)
- [It Cannot Be Right If It Was Written by AI: On Lawyers' Preferences of Documents Perceived as Authored by an LLM vs a Human](https://mlsanity.com/paper/2407.06798/) (2024)
- [ELLA: Empowering LLMs for Interpretable, Accurate and Informative Legal Advice](https://mlsanity.com/paper/2408.07137/) (2024)
- [Automating Legal Interpretation with LLMs: Retrieval, Generation, and Evaluation](https://mlsanity.com/paper/2501.01743/) (2025)
- [IslamicLegalBench: Evaluating LLMs Knowledge and Reasoning of Islamic Law Across 1,200 Years of Islamic Pluralist Legal Traditions](https://mlsanity.com/paper/2602.21226/) (2026)
- [When Geoscience Meets Generative AI and Large Language Models: Foundations, Trends, and Future Challenges](https://mlsanity.com/paper/2402.03349/) (2024)

## Abstract (arXiv)

Interpreting the meaning of legal open-textured terms is a key task of legal professionals. An important source for this interpretation is how the term was applied in previous court cases. In this paper, we evaluate the performance of GPT-4 in generating factually accurate, clear and relevant explanations of terms in legislation. We compare the performance of a baseline setup, where GPT-4 is directly asked to explain a legal term, to an augmented approach, where a legal information retrieval module is used to provide relevant context to the model, in the form of sentences from case law. We found that the direct application of GPT-4 yields explanations that appear to be of very high quality on their surface. However, detailed analysis uncovered limitations in terms of the factual accuracy of the explanations. Further, we found that the augmentation leads to improved quality, and appears to eliminate the issue of hallucination, where models invent incorrect statements. These findings open the door to the building of systems that can autonomously retrieve relevant sentences from case law and condense them into a useful explanation for legal scholars, educators or practicing lawyers alike.