mlsanityData to 7 Oct 2026

MAD-Fact: A Multi-Agent Debate Framework for Long-Form Factuality Evaluation in LLMs

arXiv 2510.22967 ↗First posted 27 Oct 2025cs.CL cs.AIcited by 4

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

The widespread adoption of Large Language Models (LLMs) raises critical concerns about the factual accuracy of their outputs, especially in high-risk domains such as biomedicine, law, and education. Existing evaluation methods for short texts often fail on long-form content due to complex reasoning chains, intertwined perspectives, and cumulative information. To address this, we propose a systematic approach integrating large-scale long-form datasets, multi-agent verification mechanisms, and weighted evaluation metrics. We construct LongHalluQA, a Chinese long-form factuality dataset; and develop MAD-Fact, a debate-based multi-agent verification system. We introduce a fact importance hierarchy to capture the varying significance of claims in long-form texts. Experiments on two benchmarks show that larger LLMs generally maintain higher factual consistency, while domestic models excel on Chinese content. Our work provides a structured framework for evaluating and enhancing factual reliability in long-form LLM outputs, guiding their safe deployment in sensitive domains.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 57 of its 69 references that are in the map. First posted on arXiv 27 Oct 2025.

Builds on

57 of 69 references.

References · 57

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  6. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  7. 2023A Survey of Large Language Modelscited by 2,537
  8. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  9. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  10. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  11. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  12. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
38 more
  1. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  2. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  3. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 752
  4. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
  5. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  6. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  7. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
  8. 2024Hallucination of Multimodal Large Language Models: A Surveycited by 380
  9. 2021A Survey on Automated Fact-Checkingcited by 376
  10. 2023A Survey of Hallucination in Large Foundation Modelscited by 365
  11. 2023HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Modelscited by 329
  12. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 308
  13. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
  14. 2023FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarioscited by 247
  15. 2024Tool Learning with Large Language Models: A Surveycited by 221
  16. 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 200
  17. 2023Large Language Models for Generative Information Extraction: A Surveycited by 183
  18. 2023Med-HALT: Medical Domain Hallucination Test for Large Language Modelscited by 156
  19. 2024Long-form factuality in large language modelscited by 146
  20. 2023Chatlaw: A Multi-Agent Legal Assistant based on a Role-Aligned Mixture-of-Experts Architecturecited by 119
  21. 2024LLM Multi-Agent Systems: Challenges and Open Problemscited by 119
  22. 2024VERISCORE: Evaluating the factuality of verifiable claims in long-form text generationcited by 114
  23. 2023Towards LLM-based Fact Verification on News Claims with a Hierarchical Step-by-Step Prompting Methodcited by 97
  24. 2023Factcheck-Bench: Fine-Grained Evaluation Benchmark for Automatic Fact-checkerscited by 86
  25. 2023Advancing Transformer Architecture in Long-Context Large Language Models: A Comprehensive Surveycited by 71
  26. 2023FELM: Benchmarking Factuality Evaluation of Large Language Modelscited by 66
  27. 2025LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directionscited by 49
  28. 2024Improving Retrieval for RAG based Question Answering Models on Financial Documentscited by 48
  29. 2024FIRE: Fact-checking with Iterative Retrieval and Verificationcited by 39
  30. 2024Chinese SimpleQA: A Chinese Factuality Evaluation for Large Language Modelscited by 39
  31. 2024LawLLM: Law Large Language Model for the US Legal Systemcited by 37
  32. 2023Evaluating Hallucinations in Chinese Large Language Modelscited by 33
  33. 2024FinTral: A Family of GPT-4 Level Multimodal Financial Large Language Modelscited by 32
  34. 2023Self-Checker: Plug-and-Play Modules for Fact-Checking with Large Language Modelscited by 31
  35. 2023The Perils & Promises of Fact-checking with Large Language Modelscited by 31
  36. 2024Self-Pluralising Culture Alignment for Large Language Modelscited by 31
  37. 2024RAGAR, Your Falsehood Radar: RAG-Augmented Reasoning for Political Fact-Checking using Multimodal Large Language Modelscited by 27
  38. 2024FactBench: A Dynamic Benchmark for In-the-Wild Language Model Factuality Evaluationcited by 23

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026When Summaries Distort Decisions: Information Fidelity in LLM-Compressed Financial Analysiscited by 2
  2. 2026FactCheck: Feasibility-aware Long-term Action Anticipation with Multi-agent Collaborationcited by 1
  3. 2026MoCA-Agent: A Market-of-Claims Code Agent for Financial and Numerical Reasoningcited by 0
  4. 2026The "10th Juror": Open-Set Standpoint Screening for Bureaucratic Bias Detectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Tool-MAD: A Multi-Agent Debate Framework for Fact Verification with Diverse Tool Augmentation and Adaptive Retrieval
  2. 2026Towards Comprehensive Stage-wise Benchmarking of Large Language Models in Fact-Checking
  3. 2025FACT-AUDIT: An Adaptive Multi-Agent Framework for Dynamic Fact-Checking Evaluation of Large Language Models
  4. 2025Understanding Inequality of LLM Fact-Checking over Geographic Regions with Agent and Retrieval models
  5. 2025CANDY: Benchmarking LLMs' Limitations and Assistive Potential in Chinese Misinformation Fact-Checking
  6. 2024Can LLMs Produce Faithful Explanations For Fact-checking? Towards Faithful Explainable Fact-Checking via Multi-Agent Debate
  7. 2026Leveraging LLM Parametric Knowledge for Fact Checking without Retrieval
  8. 2023Are Large Language Models Good Fact Checkers: A Preliminary Study