mlsanityData to 7 Oct 2026

Towards Detecting LLMs Hallucination via Markov Chain-based Multi-agent Debate Framework

arXiv 2406.03075 ↗First posted 5 Jun 2024cs.CLcited by 19

On the map: LLM agents and tool use → Medical and scientific domain agents

Abstract (arXiv)

The advent of large language models (LLMs) has facilitated the development of natural language text generation. It also poses unprecedented challenges, with content hallucination emerging as a significant concern. Existing solutions often involve expensive and complex interventions during the training process. Moreover, some approaches emphasize problem disassembly while neglecting the crucial validation process, leading to performance degradation or limited applications. To overcome these limitations, we propose a Markov Chain-based multi-agent debate verification framework to enhance hallucination detection accuracy in concise claims. Our method integrates the fact-checking process, including claim detection, evidence retrieval, and multi-agent verification. In the verification stage, we deploy multiple agents through flexible Markov Chain-based debates to validate individual claims, ensuring meticulous verification outcomes. Experimental results across three generative tasks demonstrate that our approach achieves significant improvements over baselines.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use. Builds on 35 of its 43 references that are in the map. First posted on arXiv 5 Jun 2024.

Builds on

35 of 43 references.

References · 35

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  4. 2022Emergent Abilities of Large Language Modelscited by 2,483
  5. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  6. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  7. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  8. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  9. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
  10. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  11. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
  12. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
23 more
  1. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 578
  2. 2023Fine-Grained Human Feedback Gives Better Rewards for Language Model Trainingcited by 399
  3. 2021A Survey on Automated Fact-Checkingcited by 376
  4. 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 351
  5. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 349
  6. 2023HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Modelscited by 329
  7. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 308
  8. 2023Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memorycited by 288
  9. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
  10. 2023FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarioscited by 247
  11. 2023Exploring Large Language Models for Communication Games: An Empirical Study on Werewolfcited by 238
  12. 2023Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedbackcited by 202
  13. 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 200
  14. 2023LM vs LM: Detecting Factual Errors via Cross Examinationcited by 155
  15. 2021Annotating and Modeling Fine-grained Factuality in Summarizationcited by 131
  16. 2023Examining Inter-Consistency of Large Language Models Collaboration: An In-depth Analysis via Debatecited by 94
  17. 2023SAC3: Reliable Hallucination Detection in Black-Box Language Models via Semantic-aware Cross-check Consistencycited by 89
  18. 2023Why Does ChatGPT Fall Short in Providing Truthful Answers?cited by 54
  19. 2023Self-Verification Improves Few-Shot Clinical Information Extractioncited by 52
  20. 2023Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspectioncited by 44
  21. 2023Halo: Estimation and Reduction of Hallucinations in Open-Source Weak Large Language Modelscited by 36
  22. 2023Diving Deep into Modes of Fact Hallucinations in Dialogue Systemscited by 32
  23. 2022WeCheck: Strong Factual Consistency Checker via Weakly Supervised Learningcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 9; 2026: 7.

19 papers.

Sort:

By area:

  1. 2025LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directionscited by 49
  2. 2024FIRE: Fact-checking with Iterative Retrieval and Verificationcited by 39
  3. 2024Multi-Agent Large Language Models for Conversational Task-Solvingcited by 24
  4. 2024DebUnc: Improving Large Language Model Agent Communication With Uncertainty Metricscited by 13
  5. 2026From Spark to Fire: Modeling and Mitigating Error Cascades in LLM-Based Multi-Agent Collaborationcited by 13
  6. 2025MAMM-Refine: A Recipe for Improving Faithfulness in Generation with Multi-Agent Collaborationcited by 9
  7. 2025CortexDebate: Debating Sparsely and Equally for Multi-Agent Debatecited by 6
  8. 2025HalluRNN: Mitigating Hallucinations via Recurrent Cross-Layer Reasoning in Large Vision-Language Modelscited by 4
  9. 2025FinDebate: Multi-Agent Collaborative Intelligence for Financial Analysiscited by 4
  10. 2025MAD-Fact: A Multi-Agent Debate Framework for Long-Form Factuality Evaluation in LLMscited by 4
  11. 2026Multi-Agent Debate Strategies: Survey, Taxonomy, and Challengescited by 3
  12. 2025PASS-FC: Progressive and Adaptive Search Scheme for Fact Checking of Comprehensive Claimscited by 2
7 more
  1. 2026Social Dynamics as Critical Vulnerabilities that Undermine Objective Decision-Making in LLM Collectivescited by 2
  2. 2025MAAD: Automate Software Architecture Design through Knowledge-Driven Multi-Agent Collaborationcited by 1
  3. 2025Simulating Teams with LLM Agents: Interactive 2D Environments for Studying Human-AI Dynamicscited by 0
  4. 2026Logical Consistency as a Bridge: Improving LLM Hallucination Detection via Label Constraint Modeling between Responses and Self-Judgmentscited by 0
  5. 2026Bridging Requirements and Architecture: Multi-Agent Orchestration with External Knowledge and Hierarchical Memorycited by 0
  6. 2026CWF: A Collaborative Writing Framework for Personalized and Reliable Popular Science Writingcited by 0
  7. 2026MAADBench: The Refreshable Paradigm for Anomaly Detection in Multi-Agent Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026MARCH: Multi-Agent Reinforced Self-Check for LLM Hallucination
  2. 2026Hallucination Cascade: Analyzing Error Propagation in Multi-Agent LLM Systems
  3. 2026Collective Hallucination in Multi-Agent LLMs:Modeling and Defense
  4. 2026Council Mode: A Heterogeneous Multi-Agent Consensus Framework for Reducing LLM Hallucination and Bias
  5. 2025MMD-Flagger: Leveraging Maximum Mean Discrepancy to Detect Hallucinations
  6. 2026Zero-source LLM Hallucination Detection with Human-like Criteria Probing
  7. 2026AgentHallu: Benchmarking Automated Hallucination Attribution of LLM-based Agents
  8. 2024Good Parenting is all you need -- Multi-agentic LLM Hallucination Mitigation