GANDR: Claim Auditing for Verifiable Legal Answer Generation
Not on the map. Why?
Abstract (arXiv)
In high-stakes domains such as legal practice, a language-model answer is only useful to the extent that a reader can verify each claim against the source the system cites. Current grounded-generation pipelines score the answer as a whole, so a correct conclusion can rest on fabricated or loosely matched citations and still score well. Closing this gap requires both a system built for per-claim verification and an evaluation that measures it. We introduce GANDR (Grounded ANswer DRafter), a two-agent system in which a Drafter writes an answer in a structured legal-reasoning format and a separate Critic, with the same view as a human verifier, audits each claim against its cited source and emits a per-claim audit trace on every round. We pair it with a strict correctness criterion requiring every citation to resolve to a passage the retriever returned. On a 185-item legal benchmark where all six systems share one backbone, one retrieval surface, and one citation instruction, GANDR ranks first on every primary metric, reaching 70.8% strict accuracy and leading the strongest baseline by 11.3 points (p<0.01). Reverting the protocol-anchored commit rule lowers strict accuracy by 22.7 points, and the strict lead stays positive on three further backbones, at +3.2 to +6.5 points. This lead traces to the Drafter configuration and the protocol-anchored commit, not to rewriting. Against two law-trained annotators the audit flags under-supported claims at F1 0.84 as a binary detector, while its four-way verdict labels agree only weakly and are advisory. Code is available upon request.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 26 of its 35 references that are in the map. First posted on arXiv 9 Sep 2026.
Builds on
26 of 35 references.
References · 26
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
- 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
- 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
- 2023Large Language Models are not Fair Evaluatorscited by 1,085
- 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
- 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
- 2019Passage Re-ranking with BERTcited by 645
14 more
- 2023Enabling Large Language Models to Generate Text with Citationscited by 595
- 2023LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Modelscited by 336
- 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 324
- 2023Evaluating Verifiability in Generative Search Enginescited by 303
- 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Modelscited by 283
- 2024Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Toolscited by 176
- 2024Large Legal Fictions: Profiling Legal Hallucinations in Large Language Modelscited by 150
- 2024LegalBench-RAG: A Benchmark for Retrieval-Augmented Generation in the Legal Domaincited by 118
- 2022Legal Prompting: Teaching a Language Model to Think Like a Lawyercited by 60
- 2026Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillationcited by 60
- 2024AttributionBench: How Hard is Automatic Attribution Evaluation?cited by 26
- 2023Towards Verifiable Generation: A Benchmark for Knowledge-aware Language Model Attributioncited by 24
- 2024CitaLaw: Enhancing LLM with Citations in Legal Domaincited by 16
- 2026From "Thinking" to "Justifying": Aligning High-Stakes Explainability with Professional Communication Standardscited by 1
Built on by
Not cited yet.