# Applicability of Large Language Models and Generative Models for Legal Case Judgement Summarization

arXiv: https://arxiv.org/abs/2407.12848 · HTML: https://mlsanity.com/paper/2407.12848/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 50 references that are in the map.
- First posted on arXiv 6 Jul 2024.

## Builds on (most cited first, 28 of 28)

- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Benchmarking Large Language Models for News Summarization](https://mlsanity.com/paper/2301.13848/) (2023)
- [SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization](https://mlsanity.com/paper/2111.09525/) (2021)
- [Efficient Attentions for Long Document Summarization](https://mlsanity.com/paper/2104.02112/) (2021)
- [Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies](https://mlsanity.com/paper/1804.11283/) (2018)
- [SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents](https://mlsanity.com/paper/1611.04230/) (2016)
- [BooookScore: A systematic exploration of book-length summarization in the era of LLMs](https://mlsanity.com/paper/2310.00785/) (2023)
- [Few-shot training LLMs for project-specific code-summarization](https://mlsanity.com/paper/2207.04237/) (2022)
- [Fine-tune BERT for Extractive Summarization](https://mlsanity.com/paper/1903.10318/) (2019)
- [Sentence Centrality Revisited for Unsupervised Summarization](https://mlsanity.com/paper/1906.03508/) (2019)
- [Summ^N: A Multi-Stage Summarization Framework for Long Input Dialogues and Documents](https://mlsanity.com/paper/2110.10150/) (2021)
- [How Ready are Pre-trained Abstractive Models and LLMs for Legal Case Judgement Summarization?](https://mlsanity.com/paper/2306.01248/) (2023)
- [Creating Trustworthy LLMs: Dealing with Hallucinations in Healthcare AI](https://mlsanity.com/paper/2311.01463/) (2023)
- [Extractive is not Faithful: An Investigation of Broad Unfaithfulness Problems in Extractive Summarization](https://mlsanity.com/paper/2209.03549/) (2022)
- [Legal Case Document Summarization: Extractive and Abstractive Methods and their Evaluation](https://mlsanity.com/paper/2210.07544/) (2022)
- [Incorporating Domain Knowledge for Extractive Summarization of Legal Case Documents](https://mlsanity.com/paper/2106.15876/) (2021)
- [A Novel Multi-Stage Prompting Approach for Language Agnostic MCQ Generation using GPT](https://mlsanity.com/paper/2401.07098/) (2024)
- [Exploring the Capabilities of Prompted Large Language Models in Educational and Assessment Applications](https://mlsanity.com/paper/2405.11579/) (2024)
- [How Ready Are Generative Pre-trained Large Language Models for Explaining Bengali Grammatical Errors?](https://mlsanity.com/paper/2406.00039/) (2024)
- [Fact-based Court Judgment Prediction](https://mlsanity.com/paper/2311.13350/) (2023)
- [Nonet at SemEval-2023 Task 6: Methodologies for Legal Evaluation](https://mlsanity.com/paper/2310.11049/) (2023)
- [Harnessing the Power of Prompt-based Techniques for Generating School-Level Questions using Large Language Models](https://mlsanity.com/paper/2312.01032/) (2023)
- [How Effective is GPT-4 Turbo in Generating School-Level Questions from Textbooks Based on Bloom's Revised Taxonomy?](https://mlsanity.com/paper/2406.15211/) (2024)
- [Artificial Intelligence (AI) in Legal Data Mining](https://mlsanity.com/paper/2405.14707/) (2024)

## Built on by (most cited first, 23 of 23)

- [Do Language Models Mirror Human Confidence? Exploring Psychological Insights to Address Overconfidence in LLMs](https://mlsanity.com/paper/2506.00582/) (2025)
- [Agents on the Bench: Large Language Model Based Multi Agent Framework for Trustworthy Digital Justice](https://mlsanity.com/paper/2412.18697/) (2024)
- [Improving Preference Extraction In LLMs By Identifying Latent Knowledge Through Classifying Probes](https://mlsanity.com/paper/2503.17755/) (2025)
- [Code Generation and Algorithmic Problem Solving Using Llama 3.1 405B](https://mlsanity.com/paper/2409.19027/) (2024)
- [MIRROR: A Novel Approach for the Automated Evaluation of Open-Ended Question Generation](https://mlsanity.com/paper/2410.12893/) (2024)
- [A Comprehensive Survey on Legal Summarization: Challenges and Future Directions](https://mlsanity.com/paper/2501.17830/) (2025)
- [It Cannot Be Right If It Was Written by AI: On Lawyers' Preferences of Documents Perceived as Authored by an LLM vs a Human](https://mlsanity.com/paper/2407.06798/) (2024)
- [CoPERLex: Content Planning with Event-based Representations for Legal Case Summarization](https://mlsanity.com/paper/2501.14112/) (2025)
- [When Large Language Models Meet Law: Dual-Lens Taxonomy, Technical Advances, and Ethical Governance](https://mlsanity.com/paper/2507.07748/) (2025)
- [Prompt Engineering Using GPT for Word-Level Code-Mixed Language Identification in Low-Resource Dravidian Languages](https://mlsanity.com/paper/2411.04025/) (2024)
- [Enhancing Abstractive Summarization of Scientific Papers Using Structure Information](https://mlsanity.com/paper/2505.14179/) (2025)
- [Evaluating LLM-based Approaches to Legal Citation Prediction: Domain-specific Pre-training, Fine-tuning, or RAG? A Benchmark and an Australian Law Case Study](https://mlsanity.com/paper/2412.06272/) (2024)
- [Regulatory Graphs and GenAI for Real-Time Transaction Monitoring and Compliance Explanation in Banking](https://mlsanity.com/paper/2506.01093/) (2025)
- [CourtPressGER: A German Court Decision to Press Release Summarization Dataset](https://mlsanity.com/paper/2512.09434/) (2025)
- [Advantages of Domain Knowledge Injection for Legal Document Summarization: A Case Study on Summarizing Indian Court Judgments in English and Hindi](https://mlsanity.com/paper/2602.07382/) (2026)
- [Interpretable Traffic Responsibility from Dashcam Video via Legal Multi Agent Reasoning](https://mlsanity.com/paper/2603.17930/) (2026)
- [Cancer-Answer: Empowering Cancer Care with Advanced Large Language Models](https://mlsanity.com/paper/2411.06946/) (2024)
- [CryptoLLM: Unleashing the Power of Prompted LLMs for SmartQnA and Classification of Crypto Posts](https://mlsanity.com/paper/2411.07917/) (2024)
- [Aplicacc~ao de Large Language Models na An'alise e S'intese de Documentos Jur'idicos: Uma Revis~ao de Literatura](https://mlsanity.com/paper/2504.00725/) (2025)
- [Scale-free Characteristics of Multilingual Legal Texts and the Limitations of LLMs](https://mlsanity.com/paper/2509.17367/) (2025)
- [AugAbEx: Bridging Abstractive and Extractive Legal Summarization](https://mlsanity.com/paper/2511.12290/) (2025)
- [Maat: The Agentic Legal Research Assistant for Competition Protection](https://mlsanity.com/paper/2605.27331/) (2026)
- [Reexamining zero-shot summarization: Empirical investigation of trustworthiness of LLM-summarizers](https://mlsanity.com/paper/2607.21010/) (2026)

## Abstract (arXiv)

Automatic summarization of legal case judgements, which are known to be long and complex, has traditionally been tried via extractive summarization models. In recent years, generative models including abstractive summarization models and Large language models (LLMs) have gained huge popularity. In this paper, we explore the applicability of such models for legal case judgement summarization. We applied various domain specific abstractive summarization models and general domain LLMs as well as extractive summarization models over two sets of legal case judgements from the United Kingdom (UK) Supreme Court and the Indian (IN) Supreme Court and evaluated the quality of the generated summaries. We also perform experiments on a third dataset of legal documents of a different type, Government reports from the United States (US). Results show that abstractive summarization models and LLMs generally perform better than the extractive methods as per traditional metrics for evaluating summary quality. However, detailed investigation shows the presence of inconsistencies and hallucinations in the outputs of the generative models, and we explore ways to reduce the hallucinations and inconsistencies in the summaries. Overall, the investigation suggests that further improvements are needed to enhance the reliability of abstractive models and LLMs for legal case judgement summarization. At present, a human-in-the-loop technique is more suitable for performing manual checks to identify inconsistencies in the generated summaries.