mlsanityData to 7 Oct 2026

LegalLens Shared Task 2024: Legal Violation Identification in Unstructured Text

arXiv 2410.12064 ↗First posted 15 Oct 2024cs.CL cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

This paper presents the results of the LegalLens Shared Task, focusing on detecting legal violations within text in the wild across two sub-tasks: LegalLens-NER for identifying legal violation entities and LegalLens-NLI for associating these violations with relevant legal contexts and affected individuals. Using an enhanced LegalLens dataset covering labor, privacy, and consumer protection domains, 38 teams participated in the task. Our analysis reveals that while a mix of approaches was used, the top-performing teams in both tasks consistently relied on fine-tuning pre-trained language models, outperforming legal-specific models and few-shot methods. The top-performing team achieved a 7.11% improvement in NER over the baseline, while NLI saw a more marginal improvement of 5.7%. Despite these gains, the complexity of legal texts leaves room for further advancements.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 16 of its 36 references that are in the map. First posted on arXiv 15 Oct 2024.

Builds on

16 of 36 references.

References · 16

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2019Neural Legal Judgment Prediction in Englishcited by 208
  5. 2021ContractNLI: A Dataset for Document-level Natural Language Inference for Contractscited by 145
  6. 2021Codabench: Flexible, Easy-to-Use and Reproducible Benchmarking Platformcited by 65
  7. 2023LeXFiles and LegalLAMA: Facilitating English Multinational Legal Language Model Developmentcited by 54
  8. 2018LexNLP: Natural language processing and information extraction for legal and regulatory textscited by 27
  9. 2022Named Entity Recognition in Indian court judgmentscited by 26
  10. 2022Processing Long Legal Documents with Pre-trained Transformers: Modding LegalBERT and Longformercited by 20
  11. 2022NLP-based Automated Compliance Checking of Data Processing Agreements against GDPRcited by 18
  12. 2022ClassActionPrediction: A Challenging Benchmark for Legal Judgment Prediction of Class Action Cases in the UScited by 17
4 more
  1. 2022E-NER -- An Annotated Named Entity Recognition Corpus of Legal Textcited by 13
  2. 2024LegalLens: Leveraging LLMs for Legal Violation Identification in Unstructured Textcited by 12
  3. 2022LawngNLI: A Long-Premise Benchmark for In-Domain Generalization from Short to Long Contexts and for Implication-Based Retrievalcited by 6
  4. 2023Do Language Models Learn about Legal Entity Types during Pretraining?cited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 0; 2026: 2.

5 papers.

  1. 2026CanLegalRAGBench: Evaluating Retrieval-Augmented Generation on Canadian Case Lawcited by 2
  2. 2024Bonafide at LegalLens 2024 Shared Task: Using Lightweight DeBERTa Based Encoder For Legal Violation Detection and Resolutioncited by 1
  3. 2024Augmenting Legal Decision Support Systems with LLM-based NLI for Analyzing Social Media Evidencecited by 0
  4. 2024uOttawa at LegalLens-2024: Transformer-based Classification Experimentscited by 0
  5. 2026ChildSafeAds Shared Task 2026: Commercial Content in Child-Facing YouTube Videoscited by 0