# CitaLaw: Enhancing LLM with Citations in Legal Domain

arXiv: https://arxiv.org/abs/2412.14556 · HTML: https://mlsanity.com/paper/2412.14556/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 31 references that are in the map.
- First posted on arXiv 19 Dec 2024.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Enabling Large Language Models to Generate Text with Citations](https://mlsanity.com/paper/2305.14627/) (2023)
- [MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers](https://mlsanity.com/paper/2102.01454/) (2021)
- [DISC-LawLLM: Fine-tuning Large Language Models for Intelligent Legal Services](https://mlsanity.com/paper/2309.11325/) (2023)
- [JEC-QA: A Legal-Domain Question Answering Dataset](https://mlsanity.com/paper/1911.12011/) (2019)
- [ExpertQA: Expert-Curated Questions and Attributed Answers](https://mlsanity.com/paper/2309.07852/) (2023)
- [Can GPT-3 Perform Statutory Reasoning?](https://mlsanity.com/paper/2302.06100/) (2023)
- [A Survey of Large Language Models Attribution](https://mlsanity.com/paper/2311.03731/) (2023)
- [Legal Prompting: Teaching a Language Model to Think Like a Lawyer](https://mlsanity.com/paper/2212.01326/) (2022)
- [Precedent-Enhanced Legal Judgment Prediction with LLM and Domain-Model Collaboration](https://mlsanity.com/paper/2310.09241/) (2023)
- [Explaining Legal Concepts with Augmented Large Language Models (GPT-4)](https://mlsanity.com/paper/2306.09525/) (2023)
- [Legal Judgment Prediction with Multi-Stage CaseRepresentation Learning in the Real Court Setting](https://mlsanity.com/paper/2107.05192/) (2021)
- [Explainable Legal Case Matching via Inverse Optimal Transport-based Rationale Extraction](https://mlsanity.com/paper/2207.04182/) (2022)
- [LAiW: A Chinese Legal Large Language Models Benchmark](https://mlsanity.com/paper/2310.05620/) (2023)
- [WebCiteS: Attributed Query-Focused Summarization on Chinese Web Search Results with Citations](https://mlsanity.com/paper/2403.01774/) (2024)

## Built on by (most cited first, 16 of 16)

- [PrLM: Learning Explicit Reasoning for Personalized RAG via Contrastive Reward Optimization](https://mlsanity.com/paper/2508.07342/) (2025)
- [Legal Mathematical Reasoning with LLMs: Procedural Alignment through Two-Stage Reinforcement Learning](https://mlsanity.com/paper/2504.02590/) (2025)
- [Attribution, Citation, and Quotation: A Survey of Evidence-based Text Generation with Large Language Models](https://mlsanity.com/paper/2508.15396/) (2025)
- [Large Language Model Sourcing: A Survey](https://mlsanity.com/paper/2510.10161/) (2025)
- [LLM Agents in Law: Taxonomy, Applications, and Challenges](https://mlsanity.com/paper/2601.06216/) (2026)
- [CiteVQA: Benchmarking Evidence Attribution for Trustworthy Document Intelligence](https://mlsanity.com/paper/2605.12882/) (2026)
- [ThinkEval: Practical Evaluation of Knowledge Leakage in LLM Editing using Thought-based Knowledge Graphs](https://mlsanity.com/paper/2506.01386/) (2025)
- [LexGenius: An Expert-Level Benchmark for Large Language Models in Legal General Intelligence](https://mlsanity.com/paper/2512.04578/) (2025)
- [An Explicit Syllogistic Legal Reasoning Framework for Large Language Models](https://mlsanity.com/paper/2504.04042/) (2025)
- [Legal LLM Hallucination Should Be Evaluated as Failure of Legal Warrant](https://mlsanity.com/paper/2609.17546/) (2026)
- [Evaluation of Large Language Models in Legal Applications: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/2601.15267/) (2026)
- [Aligning Large Language Model Behavior with Human Citation Preferences](https://mlsanity.com/paper/2602.05205/) (2026)
- [Seeing Before Agreeing: Aligning Multi-Agent Consensus with Visual Evidence](https://mlsanity.com/paper/2605.30698/) (2026)
- [LLMs for LLMs: A Structured Prompting Methodology for Long Legal Documents](https://mlsanity.com/paper/2509.02241/) (2025)
- [LexAgentHallu: A Hierarchical Benchmark for Profiling Hallucinations in Legal Agents](https://mlsanity.com/paper/2609.09754/) (2026)
- [GANDR: Claim Auditing for Verifiable Legal Answer Generation](https://mlsanity.com/paper/2609.10293/) (2026)

## Abstract (arXiv)

In this paper, we propose CitaLaw, the first benchmark designed to evaluate LLMs' ability to produce legally sound responses with appropriate citations. CitaLaw features a diverse set of legal questions for both laypersons and practitioners, paired with a comprehensive corpus of law articles and precedent cases as a reference pool. This framework enables LLM-based systems to retrieve supporting citations from the reference corpus and align these citations with the corresponding sentences in their responses. Moreover, we introduce syllogism-inspired evaluation methods to assess the legal alignment between retrieved references and LLM-generated responses, as well as their consistency with user questions. Extensive experiments on 2 open-domain and 7 legal-specific LLMs demonstrate that integrating legal references substantially enhances response quality. Furthermore, our proposed syllogism-based evaluation method exhibits strong agreement with human judgments.