mlsanityData to 14 Oct 2026

LexSumm and LexT5: Benchmarking and Modeling Legal Summarization Tasks in English

arXiv 2410.09527 ↗First posted 12 Oct 2024cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

In the evolving NLP landscape, benchmarks serve as yardsticks for gauging progress. However, existing Legal NLP benchmarks only focus on predictive tasks, overlooking generative tasks. This work curates LexSumm, a benchmark designed for evaluating legal summarization tasks in English. It comprises eight English legal summarization datasets, from diverse jurisdictions, such as the US, UK, EU and India. Additionally, we release LexT5, legal oriented sequence-to-sequence model, addressing the limitation of the existing BERT-style encoder-only models in the legal domain. We assess its capabilities through zero-shot probing on LegalLAMA and fine-tuning on LexSumm. Our analysis reveals abstraction and faithfulness errors even in summaries generated by zero-shot LLMs, indicating opportunities for further improvements. LexSumm benchmark and LexT5 model are available at https://github.com/TUMLegalTech/LexSumm-LexT5.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 67 of its 99 references that are in the map. First posted on arXiv 12 Oct 2024.

Builds on

67 of 99 references.

References · 67

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  5. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  6. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  7. 2020Longformer: The Long-Document Transformercited by 3,413
  8. 2019Language Models as Knowledge Bases?cited by 1,936
  9. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  10. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
  11. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
  12. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,102
38 more
  1. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
  2. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 905
  3. 2021Efficient Attentions for Long Document Summarizationcited by 345
  4. 2023LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Modelscited by 336
  5. 2021LexGLUE: A Benchmark Dataset for Legal Language Understanding in Englishcited by 310
  6. 2018Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategiescited by 294
  7. 2021The GEM Benchmark: Natural Language Generation, its Evaluation and Metricscited by 239
  8. 2021CUAD: An Expert-Annotated NLP Dataset for Legal Contract Reviewcited by 236
  9. 2021When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Datasetcited by 201
  10. 2021ContractNLI: A Dataset for Document-level Natural Language Inference for Contractscited by 148
  11. 2023BooookScore: A systematic exploration of book-length summarization in the era of LLMscited by 137
  12. 2020How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligencecited by 129
  13. 2023Unlimiformer: Long-Range Transformers with Unlimited Length Inputcited by 128
  14. 2019BillSum: A Corpus for Automatic Summarization of US Legislationcited by 117
  15. 2019JEC-QA: A Legal-Domain Question Answering Datasetcited by 116
  16. 2021Lawformer: A Pre-trained Language Model for Chinese Legal Long Documentscited by 116
  17. 2022Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Datasetcited by 113
  18. 2019Large-Scale Multi-Label Text Classification on EU Legislationcited by 104
  19. 2021MultiEURLEX -- A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfercited by 90
  20. 2018CLAUDETTE: an Automated Detector of Potentially Unfair Clauses in Online Terms of Servicecited by 73
  21. 2022Efficient Long-Text Understanding with Short-Text Modelscited by 71
  22. 2023LEXTREME: A Multi-Lingual and Multi-Task Benchmark for the Legal Domaincited by 68
  23. 2019Question Answering for Privacy Policies: Combining Computational and Legal Perspectivescited by 67
  24. 2023MultiLegalPile: A 689GB Multilingual Legal Corpuscited by 60
  25. 2021IndoNLG: Benchmark and Resources for Evaluating Indonesian Natural Language Generationcited by 57
  26. 2023LeXFiles and LegalLAMA: Facilitating English Multinational Legal Language Model Developmentcited by 54
  27. 2022FairLex: A Multilingual Benchmark for Evaluating Fairness in Legal Text Processingcited by 49
  28. 2020GLGE: A New General Language Generation Evaluation Benchmarkcited by 47
  29. 2022A Multi-Task Benchmark for Korean Legal Language Understanding and Judgement Predictioncited by 44
  30. 2022IndicNLG Benchmark: Multilingual Datasets for Diverse NLG Tasks in Indic Languagescited by 42
  31. 2021A Statutory Article Retrieval Dataset in Frenchcited by 40
  32. 2022EUR-Lex-Sum: A Multi- and Cross-lingual Dataset for Long-form Summarization in the Legal Domaincited by 40
  33. 2022Pre-trained Language Models for the Legal Domain: A Case Study on Indian Lawcited by 39
  34. 2023Natural Language Processing in the Legal Domaincited by 38
  35. 2022GEMv2: Multilingual NLG Benchmarking in a Single Line of Codecited by 36
  36. 2022ArgLegalSumm: Improving Abstractive Summarization of Legal Documents with Argument Miningcited by 25
  37. 2021LOT: A Story-Centric Benchmark for Evaluating Chinese Long Text Understanding and Generationcited by 24
  38. 2021Incorporating Domain Knowledge for Extractive Summarization of Legal Case Documentscited by 23

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 2.

8 papers.

  1. 2025Large Language Models Meet Legal Artificial Intelligence: A Surveycited by 10
  2. 2025CoPERLex: Content Planning with Event-based Representations for Legal Case Summarizationcited by 4
  3. 2025LexGenie: Automated Generation of Structured Reports for European Court of Human Rights Case Lawcited by 2
  4. 2025Evaluating Customized vs. Generalist Transformer-based Models for Legal Contract Classificationcited by 2
  5. 2025RELexED: Retrieval-Enhanced Legal Summarization with Exemplar Diversitycited by 1
  6. 2025CourtPressGER: A German Court Decision to Press Release Summarization Datasetcited by 1
  7. 2026LexLattice: Multilingual Extractive Summarization via Neural Cellular Automata on Document Hierarchiescited by 0
  8. 2026Enhancing Legal Reasoning in Pre-trained Language Models via Knowledge Graph-Guided Multi-Task Pre-trainingcited by 0