mlsanityData to 7 Oct 2026

LegalLens: Leveraging LLMs for Legal Violation Identification in Unstructured Text

arXiv 2402.04335 ↗First posted 6 Feb 2024cs.CL cs.AI cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

In this study, we focus on two main tasks, the first for detecting legal violations within unstructured textual data, and the second for associating these violations with potentially affected individuals. We constructed two datasets using Large Language Models (LLMs) which were subsequently validated by domain expert annotators. Both tasks were designed specifically for the context of class-action cases. The experimental design incorporated fine-tuning models from the BERT family and open-source LLMs, and conducting few-shot experiments using closed-source LLMs. Our results, with an F1-score of 62.69% (violation identification) and 81.02% (associating victims), show that our datasets and setups can be used for both tasks. Finally, we publicly release the datasets and the code used for the experiments in order to advance further research in the area of legal natural language processing (NLP).

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 33 of its 54 references that are in the map. First posted on arXiv 6 Feb 2024.

Builds on

33 of 54 references.

References · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  7. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  8. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  9. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  10. 2020Longformer: The Long-Document Transformercited by 3,362
  11. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  12. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
21 more
  1. 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 705
  2. 2019How to Fine-Tune BERT for Text Classification?cited by 409
  3. 2022Is GPT-3 a Good Data Annotator?cited by 208
  4. 2020HoVer: A Dataset for Many-Hop Fact Extraction And Claim Verificationcited by 199
  5. 2021ContractNLI: A Dataset for Document-level Natural Language Inference for Contractscited by 145
  6. 2020Training Question Answering Models From Synthetic Datacited by 95
  7. 2023MultiLegalPile: A 689GB Multilingual Legal Corpuscited by 59
  8. 2023LeXFiles and LegalLAMA: Facilitating English Multinational Legal Language Model Developmentcited by 54
  9. 2023Explaining Legal Concepts with Augmented Large Language Models (GPT-4)cited by 39
  10. 2022LINGUIST: Language Model Instruction Tuning to Generate Annotated Utterances for Intent Classification and Slot Taggingcited by 29
  11. 2021On the Ethical Limits of Natural Language Processing on Legal Textcited by 26
  12. 2022Named Entity Recognition in Indian court judgmentscited by 26
  13. 2020A Dataset of German Legal Documents for Named Entity Recognitioncited by 24
  14. 2022CLASP: Few-Shot Cross-Lingual Data Augmentation for Semantic Parsingcited by 21
  15. 2022Processing Long Legal Documents with Pre-trained Transformers: Modding LegalBERT and Longformercited by 20
  16. 2022NLP-based Automated Compliance Checking of Data Processing Agreements against GDPRcited by 18
  17. 2022ClassActionPrediction: A Challenging Benchmark for Legal Judgment Prediction of Class Action Cases in the UScited by 17
  18. 2021Response Generation with Context-Aware Prompt Learningcited by 12
  19. 2023Prototyping the use of Large Language Models (LLMs) for adult learning content creation at scalecited by 12
  20. 2022Time-aware Prompting for Text Generationcited by 10
  21. 2020Named Entity Recognition in the Legal Domain using a Pointer Generator Networkcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 2; 2026: 2.

12 papers.

  1. 2024LawInstruct: A Resource for Studying Language Model Adaptation to the Legal Domaincited by 10
  2. 2024LegalLens Shared Task 2024: Legal Violation Identification in Unstructured Textcited by 5
  3. 2024llmNER: (Zero|Few)-Shot Named Entity Recognition, Exploiting the Power of Large Language Modelscited by 4
  4. 2024Developing a Pragmatic Benchmark for Assessing Korean Legal Language Understanding in Large Language Modelscited by 4
  5. 2025On the Robustness of Verbal Confidence of LLMs in Adversarial Attackscited by 3
  6. 2024Bonafide at LegalLens 2024 Shared Task: Using Lightweight DeBERTa Based Encoder For Legal Violation Detection and Resolutioncited by 1
  7. 2026ViLegalNLI: Natural Language Inference for Vietnamese Legal Textscited by 1
  8. 2024Automatic Information Extraction From Employment Tribunal Judgements Using Large Language Modelscited by 0
  9. 2024Augmenting Legal Decision Support Systems with LLM-based NLI for Analyzing Social Media Evidencecited by 0
  10. 2024uOttawa at LegalLens-2024: Transformer-based Classification Experimentscited by 0
  11. 2025LLM-Assisted Formalization Enables Deterministic Detection of Statutory Inconsistency in the Internal Revenue Codecited by 0
  12. 2026LePREC: Reasoning as Classification over Structured Factors for Assessing Relevance of Legal Issuescited by 0