L-MARS: Legal Multi-Agent System with Agentic Search and Citation-Faithfulness Audit
On the map: LLM agents and tool use → Social simulation and multi-agent debate
Abstract (arXiv)
Large language models are increasingly deployed for legal question answering, where evaluations typically focus on multiple-choice accuracy. This measure overlooks a common failure: whether the citation source attached to an answer exists and supports the rule the system attributes to it. We present L-MARS, an open multi-agent legal QA system with agentic search and judge-driven evidence checks, and audit it claim by claim against its cited source. Each atomic claim is labelled with a six-class taxonomy and scored with strict-ALCE under cross-provider judging, where the answerer and verifier come from different model families. On a stratified 100-question Bar Exam audit, retrieval barely moves accuracy, yet the multi-turn judge loop lifts strict citation F1 from 0.13 (naive RAG) to 0.25 and cuts the no-citation rate from 34% to 13%. We further introduce Faith-Search, a post-draft step that re-verifies and repairs unreachable citations; it drops the unreachable rate below 1% but does not improve F1 over the multi-turn loop, so we report it as a targeted reachability intervention rather than a faithfulness breakthrough. A 50-question LegalSearchQA case study confirms the picture: retrieve-then-draft pipelines saturate near 0.75 citation F1, while a single-agent web-search baseline collapses to 0.22 under external audit.
Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 24 of its 43 references that are in the map. First posted on arXiv 31 Aug 2025.
Builds on
24 of 43 references.
References · 24
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2022Survey of Hallucination in Natural Language Generationcited by 2,075
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
- 2022Language Models (Mostly) Know What They Knowcited by 1,856
- 2017Selective Classification for Deep Neural Networkscited by 852
- 2023Enabling Large Language Models to Generate Text with Citationscited by 595
- 2025Search-o1: Agentic Search-Enhanced Large Reasoning Modelscited by 433
- 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 335
12 more
- 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 318
- 2021LexGLUE: A Benchmark Dataset for Legal Language Understanding in Englishcited by 310
- 2025Agentic Retrieval-Augmented Generation: A Survey on Agentic RAGcited by 249
- 2024Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Toolscited by 176
- 2024Large Legal Fictions: Profiling Legal Hallucinations in Large Language Modelscited by 150
- 2025AbstentionBench: Reasoning LLMs Fail on Unanswerable Questionscited by 145
- 2023LawBench: Benchmarking Legal Knowledge of Large Language Modelscited by 117
- 2022Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Datasetcited by 113
- 2022Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Modelscited by 112
- 2025LEXam: Benchmarking Legal Reasoning on 340 Law Examscited by 54
- 2025A Reasoning-Focused Legal Retrieval Benchmarkcited by 37
- 2022Case law retrieval: problems, methods, challenges and evaluations in the last 20 yearscited by 10
Built on by
Citing papers per year: 2025: 2; 2026: 9.
11 papers.
- 2026τ-Knowledge: Evaluating Conversational Agents over Unstructured Knowledgecited by 39
- 2026SAGE: Benchmarking and Improving Retrieval for Deep Research Agentscited by 7
- 2025AIReg-Bench: Benchmarking Language Models That Assess AI Regulation Compliancecited by 6
- 2026LLM Agents in Law: Taxonomy, Applications, and Challengescited by 5
- 2026LawThinker: A Deep Research Legal Agent in Dynamic Environmentscited by 4
- 2026ClawTrace: Cost-Aware Tracing for LLM Agent Skill Distillationcited by 4
- 2025Mitigating Hallucination in Large Language Models (LLMs): An Application-Oriented Survey on RAG, Reasoning, and Agentic Systemscited by 2
- 2026Decompose-and-Refine: Structured Legal Question Answering with Parametric Retrievalcited by 0
- 2026Maat: The Agentic Legal Research Assistant for Competition Protectioncited by 0
- 2026A Matter of TASTE: Improving Coverage and Difficulty of Agent Benchmarkscited by 0
- 2026LabourCrew: A Multi-Agent RAG Framework for Trustworthy Adversarial Deliberation and Statutory Reasoning over Labour Lawcited by 0
- 2026LabourCrew: A Multi-Agent RAG Framework for Trustworthy Adversarial Deliberation and Statutory Reasoning over Labour Law18 Aug 2026
- 2026A Matter of TASTE: Improving Coverage and Difficulty of Agent Benchmarks27 May 2026
- 2026Maat: The Agentic Legal Research Assistant for Competition Protection26 May 2026
- 2026Decompose-and-Refine: Structured Legal Question Answering with Parametric Retrieval23 May 2026
- 2026ClawTrace: Cost-Aware Tracing for LLM Agent Skill Distillation26 Apr 2026
- 2026τ-Knowledge: Evaluating Conversational Agents over Unstructured Knowledge4 Mar 2026
- 2026LawThinker: A Deep Research Legal Agent in Dynamic Environments12 Feb 2026
- 2026SAGE: Benchmarking and Improving Retrieval for Deep Research Agents5 Feb 2026
- 2026LLM Agents in Law: Taxonomy, Applications, and Challenges8 Jan 2026
- 2025Mitigating Hallucination in Large Language Models (LLMs): An Application-Oriented Survey on RAG, Reasoning, and Agentic Systems28 Oct 2025
- 2025AIReg-Bench: Benchmarking Language Models That Assess AI Regulation Compliance1 Oct 2025
Similar, not cited
Close in wording, no citation either way
- 2026Legal-R1: Agentic Retrieval for Evidence-Based Legal Reasoning
- 2026L-MAD: A Systematic Evaluation of Multi-Agent Debate Structures in Legal Reasoning
- 2026LawCompass: Navigating from Legal QA to Multi-Agent Deep Research with Grounded Evidence
- 2026Courtroom-Style Multi-Agent Debate with Progressive RAG and Role-Switching for Controversial Claim Verification
- 2025Hybrid Retrieval-Augmented Generation Agent for Trustworthy Legal Question Answering in Judicial Forensics
- 2025PAKTON: A Multi-Agent Framework for Question Answering in Long Legal Agreements
- 2026Tool-MAD: A Multi-Agent Debate Framework for Fact Verification with Diverse Tool Augmentation and Adaptive Retrieval
- 2025LegalWiz: A Multi-Agent Generation Framework for Contradiction Detection in Legal Documents