# LegalLens Shared Task 2024: Legal Violation Identification in Unstructured Text

arXiv: https://arxiv.org/abs/2410.12064 · HTML: https://mlsanity.com/paper/2410.12064/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 36 references that are in the map.
- First posted on arXiv 15 Oct 2024.

## Builds on (most cited first, 16 of 16)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Neural Legal Judgment Prediction in English](https://mlsanity.com/paper/1906.02059/) (2019)
- [ContractNLI: A Dataset for Document-level Natural Language Inference for Contracts](https://mlsanity.com/paper/2110.01799/) (2021)
- [Codabench: Flexible, Easy-to-Use and Reproducible Benchmarking Platform](https://mlsanity.com/paper/2110.05802/) (2021)
- [LeXFiles and LegalLAMA: Facilitating English Multinational Legal Language Model Development](https://mlsanity.com/paper/2305.07507/) (2023)
- [LexNLP: Natural language processing and information extraction for legal and regulatory texts](https://mlsanity.com/paper/1806.03688/) (2018)
- [Named Entity Recognition in Indian court judgments](https://mlsanity.com/paper/2211.03442/) (2022)
- [Processing Long Legal Documents with Pre-trained Transformers: Modding LegalBERT and Longformer](https://mlsanity.com/paper/2211.00974/) (2022)
- [NLP-based Automated Compliance Checking of Data Processing Agreements against GDPR](https://mlsanity.com/paper/2209.09722/) (2022)
- [ClassActionPrediction: A Challenging Benchmark for Legal Judgment Prediction of Class Action Cases in the US](https://mlsanity.com/paper/2211.00582/) (2022)
- [E-NER -- An Annotated Named Entity Recognition Corpus of Legal Text](https://mlsanity.com/paper/2212.09306/) (2022)
- [LegalLens: Leveraging LLMs for Legal Violation Identification in Unstructured Text](https://mlsanity.com/paper/2402.04335/) (2024)
- [LawngNLI: A Long-Premise Benchmark for In-Domain Generalization from Short to Long Contexts and for Implication-Based Retrieval](https://mlsanity.com/paper/2212.03222/) (2022)
- [Do Language Models Learn about Legal Entity Types during Pretraining?](https://mlsanity.com/paper/2310.13092/) (2023)

## Built on by (most cited first, 5 of 5)

- [CanLegalRAGBench: Evaluating Retrieval-Augmented Generation on Canadian Case Law](https://mlsanity.com/paper/2605.30497/) (2026)
- [Bonafide at LegalLens 2024 Shared Task: Using Lightweight DeBERTa Based Encoder For Legal Violation Detection and Resolution](https://mlsanity.com/paper/2410.22977/) (2024)
- [Augmenting Legal Decision Support Systems with LLM-based NLI for Analyzing Social Media Evidence](https://mlsanity.com/paper/2410.15990/) (2024)
- [uOttawa at LegalLens-2024: Transformer-based Classification Experiments](https://mlsanity.com/paper/2410.21139/) (2024)
- [ChildSafeAds Shared Task 2026: Commercial Content in Child-Facing YouTube Videos](https://mlsanity.com/paper/2608.19165/) (2026)

## Abstract (arXiv)

This paper presents the results of the LegalLens Shared Task, focusing on detecting legal violations within text in the wild across two sub-tasks: LegalLens-NER for identifying legal violation entities and LegalLens-NLI for associating these violations with relevant legal contexts and affected individuals. Using an enhanced LegalLens dataset covering labor, privacy, and consumer protection domains, 38 teams participated in the task. Our analysis reveals that while a mix of approaches was used, the top-performing teams in both tasks consistently relied on fine-tuning pre-trained language models, outperforming legal-specific models and few-shot methods. The top-performing team achieved a 7.11% improvement in NER over the baseline, while NLI saw a more marginal improvement of 5.7%. Despite these gains, the complexity of legal texts leaves room for further advancements.