mlsanityData to 7 Oct 2026

CitaLaw: Enhancing LLM with Citations in Legal Domain

arXiv 2412.14556 ↗First posted 19 Dec 2024cs.CLcited by 16

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose CitaLaw, the first benchmark designed to evaluate LLMs' ability to produce legally sound responses with appropriate citations. CitaLaw features a diverse set of legal questions for both laypersons and practitioners, paired with a comprehensive corpus of law articles and precedent cases as a reference pool. This framework enables LLM-based systems to retrieve supporting citations from the reference corpus and align these citations with the corresponding sentences in their responses. Moreover, we introduce syllogism-inspired evaluation methods to assess the legal alignment between retrieved references and LLM-generated responses, as well as their consistency with user questions. Extensive experiments on 2 open-domain and 7 legal-specific LLMs demonstrate that integrating legal references substantially enhances response quality. Furthermore, our proposed syllogism-based evaluation method exhibits strong agreement with human judgments.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 17 of its 31 references that are in the map. First posted on arXiv 19 Dec 2024.

Builds on

17 of 31 references.

References · 17

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  4. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  5. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 466
  6. 2023DISC-LawLLM: Fine-tuning Large Language Models for Intelligent Legal Servicescited by 117
  7. 2019JEC-QA: A Legal-Domain Question Answering Datasetcited by 115
  8. 2023ExpertQA: Expert-Curated Questions and Attributed Answerscited by 96
  9. 2023Can GPT-3 Perform Statutory Reasoning?cited by 71
  10. 2023A Survey of Large Language Models Attributioncited by 71
  11. 2022Legal Prompting: Teaching a Language Model to Think Like a Lawyercited by 60
  12. 2023Precedent-Enhanced Legal Judgment Prediction with LLM and Domain-Model Collaborationcited by 52
5 more
  1. 2023Explaining Legal Concepts with Augmented Large Language Models (GPT-4)cited by 39
  2. 2021Legal Judgment Prediction with Multi-Stage CaseRepresentation Learning in the Real Court Settingcited by 24
  3. 2022Explainable Legal Case Matching via Inverse Optimal Transport-based Rationale Extractioncited by 21
  4. 2023LAiW: A Chinese Legal Large Language Models Benchmarkcited by 9
  5. 2024WebCiteS: Attributed Query-Focused Summarization on Chinese Web Search Results with Citationscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 8; 2026: 8.

16 papers.

Sort:

By area:

  1. 2025PrLM: Learning Explicit Reasoning for Personalized RAG via Contrastive Reward Optimizationcited by 8
  2. 2025Legal Mathematical Reasoning with LLMs: Procedural Alignment through Two-Stage Reinforcement Learningcited by 7
  3. 2025Attribution, Citation, and Quotation: A Survey of Evidence-based Text Generation with Large Language Modelscited by 7
  4. 2025Large Language Model Sourcing: A Surveycited by 5
  5. 2026LLM Agents in Law: Taxonomy, Applications, and Challengescited by 5
  6. 2026CiteVQA: Benchmarking Evidence Attribution for Trustworthy Document Intelligencecited by 5
  7. 2025ThinkEval: Practical Evaluation of Knowledge Leakage in LLM Editing using Thought-based Knowledge Graphscited by 4
  8. 2025LexGenius: An Expert-Level Benchmark for Large Language Models in Legal General Intelligencecited by 3
  9. 2025An Explicit Syllogistic Legal Reasoning Framework for Large Language Modelscited by 2
  10. 2026Evaluation of Large Language Models in Legal Applications: Challenges, Methods, and Future Directionscited by 1
  11. 2026Aligning Large Language Model Behavior with Human Citation Preferencescited by 1
  12. 2026Seeing Before Agreeing: Aligning Multi-Agent Consensus with Visual Evidencecited by 1
4 more
  1. 2026Legal LLM Hallucination Should Be Evaluated as Failure of Legal Warrantcited by 1
  2. 2025LLMs for LLMs: A Structured Prompting Methodology for Long Legal Documentscited by 0
  3. 2026LexAgentHallu: A Hierarchical Benchmark for Profiling Hallucinations in Legal Agentscited by 0
  4. 2026GANDR: Claim Auditing for Verifiable Legal Answer Generationcited by 0