mlsanityData to 7 Oct 2026

Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools

arXiv 2405.20362 ↗First posted 30 May 2024cs.CL cs.CYcited by 170

Not on the map. Why?

Cited by 38 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Legal practice has witnessed a sharp rise in products incorporating artificial intelligence (AI). Such tools are designed to assist with a wide range of core legal tasks, from search and summarization of caselaw to document drafting. But the large language models used in these tools are prone to "hallucinate," or make up false information, making their use risky in high-stakes domains. Recently, certain legal research providers have touted methods such as retrieval-augmented generation (RAG) as "eliminating" (Casetext, 2023) or "avoid[ing]" hallucinations (Thomson Reuters, 2023), or guaranteeing "hallucination-free" legal citations (LexisNexis, 2023). Because of the closed nature of these systems, systematically assessing these claims is challenging. In this article, we design and report on the first preregistered empirical evaluation of AI-driven legal research tools. We demonstrate that the providers' claims are overstated. While hallucinations are reduced relative to general-purpose chatbots (GPT-4), we find that the AI research tools made by LexisNexis (Lexis+ AI) and Thomson Reuters (Westlaw AI-Assisted Research and Ask Practical Law AI) each hallucinate between 17% and 33% of the time. We also document substantial differences between systems in responsiveness and accuracy. Our article makes four key contributions. It is the first to assess and report the performance of RAG-based proprietary legal AI tools. Second, it introduces a comprehensive, preregistered dataset for identifying and understanding vulnerabilities in these systems. Third, it proposes a clear typology for differentiating between hallucinations and accurate legal responses. Last, it provides evidence to inform the responsibilities of legal professionals in supervising and verifying AI outputs, which remains a central open question for the responsible integration of AI into law.

Key facts. Cited by 170 papers in the map (data through 7 Oct 2026). Builds on 41 of its 145 references that are in the map. First posted on arXiv 30 May 2024.

Builds on

41 of 145 references.

References · 41

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  7. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  8. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  9. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  10. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  11. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  12. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
29 more
  1. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  2. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  3. 2023Towards Understanding Sycophancy in Language Modelscited by 988
  4. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  5. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  6. 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 515
  7. 2016How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generationcited by 356
  8. 2021AI and the Everything in the Whole Wide World Benchmarkcited by 340
  9. 2023LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Modelscited by 328
  10. 2023Benchmarking Large Language Models in Retrieval-Augmented Generationcited by 307
  11. 2024The Power of Noise: Redefining Retrieval for RAG Systemscited by 275
  12. 2023How is ChatGPT's behavior changing over time?cited by 265
  13. 2021When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Datasetcited by 199
  14. 2024Large Legal Fictions: Profiling Legal Hallucinations in Large Language Modelscited by 144
  15. 2024Seven Failure Points When Engineering a Retrieval Augmented Generation Systemcited by 138
  16. 2023Investigating Data Contamination in Modern Benchmarks for Large Language Modelscited by 136
  17. 2019Unifying Human and Statistical Evaluation for Natural Language Generationcited by 127
  18. 2022Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answeringcited by 126
  19. 2023Do Language Models Know When They're Hallucinating References?cited by 115
  20. 2023Calibrated Language Models Must Hallucinatecited by 114
  21. 2022Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Datasetcited by 113
  22. 2022Task-aware Retrieval with Instructionscited by 107
  23. 2023Task Contamination: Language Models May Not Be Few-Shot Anymorecited by 98
  24. 2022The Fallacy of AI Functionalitycited by 88
  25. 2022Outsider Oversight: Designing a Third Party Audit Ecosystem for AI Governancecited by 67
  26. 2022Human Evaluation of Conversations is an Open Problem: comparing the sensitivity of various methods for evaluating dialogue agentscited by 62
  27. 2023The Cambridge Law Corpus: A Dataset for Legal AI Researchcited by 14
  28. 2024Promises and pitfalls of artificial intelligence for legal applicationscited by 14
  29. 2024Caveat Lector: Large Language Models in Legal Practicecited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 28; 2025: 68; 2026: 74.

170 papers.

Sort:

By area:

  1. 2024RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systemscited by 115
  2. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 100
  3. 2024ReDeEP: Detecting Hallucination in Retrieval-Augmented Generation via Mechanistic Interpretabilitycited by 100
  4. 2025Agentic AI: A Comprehensive Survey of Architectures, Applications, and Future Directionscited by 46
  5. 2024Reasoning-Enhanced Healthcare Predictions with Knowledge Graph Community Retrievalcited by 45
  6. 2025A Reasoning-Focused Legal Retrieval Benchmarkcited by 37
  7. 2025LettuceDetect: A Hallucination Detection Framework for RAG Applicationscited by 32
  8. 2025Towards Robust Legal Reasoning: Harnessing Logical LLMs in Lawcited by 30
  9. 2025A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directionscited by 30
  10. 2024HyPA-RAG: A Hybrid Parameter Adaptive Retrieval-Augmented Generation System for AI Legal and Policy Applicationscited by 22
  11. 2024CLERC: A Dataset for Legal Case Retrieval and Retrieval-Augmented Analysis Generationcited by 18
  12. 2024LRP4RAG: Detecting Hallucinations in Retrieval-Augmented Generation via Layer-wise Relevance Propagationcited by 17
38 more
  1. 2025CondAmbigQA: A Benchmark and Dataset for Conditional Ambiguous Question Answeringcited by 15
  2. 2025Explainable Rule Application via Structured Prompting: A Neural-Symbolic Approachcited by 14
  3. 2025HalluciNot: Hallucination Detection Through Context and Common Knowledge Verificationcited by 13
  4. 2025Towards Reliable Retrieval in RAG Systems for Large Legal Datasetscited by 12
  5. 2024Problem Solving Through Human-AI Preference-Based Cooperationcited by 11
  6. 2025ACORD: An Expert-Annotated Retrieval Dataset for Legal Contract Draftingcited by 11
  7. 2025L-MARS: Legal Multi-Agent System with Agentic Search and Citation-Faithfulness Auditcited by 11
  8. 2025Mitigating Hallucination in Large Vision-Language Models via Adaptive Attention Calibrationcited by 10
  9. 2025RAG-KG-IL: A Multi-Agent Hybrid Framework for Reducing Hallucinations and Enhancing LLM Reasoning through RAG and Incremental Knowledge Graph Learning Integrationcited by 9
  10. 2025Classifying and Addressing the Diversity of Errors in Retrieval-Augmented Generation Systemscited by 9
  11. 2025Mitigating Manipulation and Enhancing Persuasion: A Reflective Multi-Agent Approach for Legal Argument Generationcited by 8
  12. 2024Mind the Data Gap: Bridging LLMs to Enterprise Data Integrationcited by 7
  13. 2025Challenges for Generative AI in Legal Reasoningcited by 7
  14. 2025Linguistic Nepotism: Trading-off Quality for Language Preference in Multilingual RAGcited by 7
  15. 2025LegalWiz: A Multi-Agent Generation Framework for Contradiction Detection in Legal Documentscited by 7
  16. 2025Moloch's Bargain: Emergent Misalignment When LLMs Compete for Audiencescited by 7
  17. 2026HalluHard: A Hard Multi-Turn Hallucination Benchmarkcited by 7
  18. 2026SoK: Agentic Retrieval-Augmented Generation (RAG): Taxonomy, Architectures, Evaluation, and Research Directionscited by 7
  19. 2024CoverBench: A Challenging Benchmark for Complex Claim Verificationcited by 6
  20. 2025A Comprehensive Survey on Legal Summarization: Challenges and Future Directionscited by 6
  21. 2025Incorporating Legal Structure in Retrieval-Augmented Generation: A Case Study on Copyright Fair Usecited by 6
  22. 2025Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Modelscited by 6
  23. 2025CoCoLex: Confidence-guided Copy-based Decoding for Grounded Legal Text Generationcited by 6
  24. 2025Do LLMs Truly Understand When a Precedent Is Overruled?cited by 6
  25. 2025Principled Context Engineering for RAG: Statistical Guarantees via Conformal Predictioncited by 6
  26. 2025Toward Faithful Retrieval-Augmented Generation with Sparse Autoencoderscited by 6
  27. 2026FACTUM: Mechanistic Detection of Citation Hallucination in Long-Form RAGcited by 6
  28. 2024Luna: An Evaluation Foundation Model to Catch Language Model Hallucinations with High Accuracy and Low Costcited by 5
  29. 2024It Cannot Be Right If It Was Written by AI: On Lawyers' Preferences of Documents Perceived as Authored by an LLM vs a Humancited by 5
  30. 2024GroUSE: A Benchmark to Evaluate Evaluators in Grounded Question Answeringcited by 5
  31. 2024Belief in the Machine: Investigating Epistemological Blind Spots of Language Modelscited by 5
  32. 2025LRAGE: Legal Retrieval Augmented Generation Evaluation Toolcited by 5
  33. 2025LegalSearchLM: Rethinking Legal Case Retrieval as Legal Elements Generationcited by 5
  34. 2025AIReg-Bench: Benchmarking Language Models That Assess AI Regulation Compliancecited by 5
  35. 2026LLM Agents in Law: Taxonomy, Applications, and Challengescited by 5
  36. 2024Gaps or Hallucinations? Gazing into Machine-Generated Legal Analysis for Fine-grained Text Evaluationscited by 4
  37. 2024Developing a Pragmatic Benchmark for Assessing Korean Legal Language Understanding in Large Language Modelscited by 4
  38. 2024LAR-ECHR: A New Legal Argument Reasoning Task and Dataset for Cases of the European Court of Human Rightscited by 4

50 most cited of 170 All 170 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Why Avoid Generative Legal AI Systems? Hallucination, Overreliance, and their Impact on Explainability ↗
  2. 2026LexAgentHallu: A Hierarchical Benchmark for Profiling Hallucinations in Legal Agents
  3. 2025Inteligencia Artificial jur'idica y el desaf'io de la veracidad: an'alisis de alucinaciones, optimizaci'on de RAG y principios para una integraci'on responsable
  4. 2024RAGged Edges: The Double-Edged Sword of Retrieval-Augmented Chatbots
  5. 2025Fine-tuning Large Language Models for Improving Factuality in Legal Question Answering
  6. 2026Ordinary, Reasonable Chatbots: Do AI Models Track Human Legal Judgments?
  7. 2026Steering the Verifiability of Multimodal AI Hallucinations
  8. 2025Hallucination Mitigation using Agentic AI Natural Language-Based Frameworks