mlsanityData to 7 Oct 2026

Talk Isn't Always Cheap: Understanding Failure Modes in Multi-Agent Debate

arXiv 2509.05396 ↗First posted 5 Sep 2025cs.CL cs.AI cs.MAcited by 63

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

While multi-agent debate has been proposed as a promising strategy for improving AI reasoning ability, we find that debate can sometimes be harmful rather than helpful. Prior work has primarily focused on debates within homogeneous groups of agents, whereas we explore how diversity in model capabilities influences the dynamics and outcomes of multi-agent interactions. Through a series of experiments, we demonstrate that debate can lead to a decrease in accuracy over time - even in settings where stronger (i.e., more capable) models outnumber their weaker counterparts. Our analysis reveals that models frequently shift from correct to incorrect answers in response to peer reasoning, favoring agreement over challenging flawed reasoning. We perform additional experiments investigating various potential contributing factors to these harmful shifts - including sycophancy, social conformity, and model and task type. These results highlight important failure modes in the exchange of reasons during multi-agent debate, suggesting that naive applications of debate may cause performance degradation when agents are neither incentivised nor adequately equipped to resist persuasive but incorrect reasoning.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 25 of its 35 references that are in the map. First posted on arXiv 5 Sep 2025.

Builds on

25 of 35 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Multiagent Finetuning: Self Improvement with Diverse Reasoning Chains10 Jan
  2. 2024Enhancing LLM Reasoning with Multi-Path Collaborative Reactive and Reflection agents31 Dec
  3. 2024Mixture-of-Agents Enhances Large Language Model Capabilities7 Jun
  4. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate30 May

Other references · 21

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  5. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  6. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  7. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  8. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  9. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
  10. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
  11. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 424
  12. 2018AI safety via debatecited by 384
9 more
  1. 2023A Survey of Reinforcement Learning from Human Feedbackcited by 259
  2. 2024Debating with More Persuasive LLMs Leads to More Truthful Answerscited by 255
  3. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 230
  4. 2023Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedbackcited by 202
  5. 2024On scalable oversight with weak LLMs judging strong LLMscited by 94
  6. 2024MultiAgent Collaboration Attack: Investigating Adversarial Attacks in Large Language Model Collaborations via Debatecited by 54
  7. 2024ACC-Collab: An Actor-Critic Approach to Multi-Agent LLM Collaborationcited by 22
  8. 2025LLM-Powered Decentralized Generative Agents with Adaptive Hierarchical Knowledge Graph for Cooperative Planningcited by 17
  9. 2025When Persuasion Overrides Truth in Multi-Agent LLM Debates: Introducing a Confidence-Weighted Persuasion Override Rate (CW-POR)cited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 54.

63 papers.

Sort:

By area:

  1. 2025Peacemaker or Troublemaker: How Sycophancy Shapes Multi-Agent Debatecited by 29
  2. 2026Demystifying Multi-Agent Debate: The Role of Confidence and Diversitycited by 17
  3. 2025The Collaboration Gap: Exploration and Benchmarking of Open-World Agentic Cooperationcited by 11
  4. 2026Reasoning Models Generate Societies of Thoughtcited by 11
  5. 2026Thought Virus: Viral Misalignment via Subliminal Prompting in Multi-Agent Systemscited by 7
  6. 2026On the Reliability Limits of LLM-Based Multi-Agent Planningcited by 7
  7. 2026Diversity Collapse in Multi-Agent LLM Systems: Structural Coupling and Collective Failure in Open-Ended Idea Generationcited by 7
  8. 2025AgentAsk: Multi-Agent Systems Need to Askcited by 6
  9. 2026Not All Flips Are Conformity: Decomposing Stance Convergence in Multi-Agent LLM Debatecited by 6
  10. 2025Be My Eyes: Extending Large Language Models to New Modalities Through Multi-Agent Collaborationcited by 5
  11. 2026DynaDebate: Breaking Homogeneity in Multi-Agent Debate with Dynamic Path Generationcited by 5
  12. 2026Conversation for Non-verifiable Learning: Self-Evolving LLMs through Meta-Evaluationcited by 5
38 more
  1. 2026Epistemic Gain, Aleatoric Cost: Uncertainty Decomposition in Multi-Agent Debate for Math Reasoningcited by 5
  2. 2026Heterogeneous Debate Engine: Identity-Grounded Cognitive Architecture for Resilient LLM-Based Ethical Tutoringcited by 5
  3. 2026Too Polite to Disagree: Understanding Sycophancy Propagation in Multi-Agent Systemscited by 5
  4. 2025When and Why Does Multi-Agent Debate Fail and Does It Really Underperform?cited by 4
  5. 2026The Illusion of Multi-Agent Advantagecited by 4
  6. 2026Delayed Verification Destabilizes Multi-Agent LLM Belief: Instability Thresholds and Optimal Corrector Placementcited by 4
  7. 2025Natural Language Tools: A Natural Language Approach to Tool Calling In Large Language Agentscited by 3
  8. 2026AgentCollabBench: Diagnosing When Good Agents Make Bad Collaboratorscited by 3
  9. 2026The Ringelmann Effect in Multi-Agent LLM Systems: A Scaling Law for Effective Team Sizecited by 3
  10. 2026SLMJury: Can Small Language Models Judge as Well as Large Ones?cited by 3
  11. 2025Reliable agent engineering should integrate machine-compatible organizational principlescited by 2
  12. 2026The Cost of Consensus: Isolated Self-Correction Prevails Over Unguided Homogeneous Multi-Agent Debatecited by 2
  13. 2026Coordination as an Architectural Layer for LLM-Based Multi-Agent Systemscited by 2
  14. 2026Not Just RLHF: Why Alignment Alone Won't Fix Multi-Agent Sycophancycited by 2
  15. 2026Flag Game: A Toy Model for Mechanistic Swarm Interpretabilitycited by 2
  16. 2025Graph of Agents: Principled Long Context Modeling by Emergent Multi-Agent Collaborationcited by 1
  17. 2025Convergence dynamics of Agent-to-Agent Interactions with Misaligned objectivescited by 1
  18. 2026M3MAD-Bench: Multi-Dimensional Evaluation of Multi-Agent Debate Across Domains and Modalitiescited by 1
  19. 2026Belief-Driven Multi-Agent Collaboration via Approximate Perfect Bayesian Equilibrium for Social Simulationcited by 1
  20. 2026MAVEN: Multi-Agent Verification-Elaboration Network with In-Step Epistemic Auditingcited by 1
  21. 2026GAMBIT: A Three-Mode Benchmark for Adversarial Robustness in Multi-Agent LLM Collectivescited by 1
  22. 2026An Enigma of Artificial Reason: Investigating the Production-Evaluation Gap in Large Reasoning Modelscited by 1
  23. 2026The Deliberative Illusion: Diagnosing Factual Attrition and Stance Homogenization in Multi-Agent LLM Deliberationcited by 1
  24. 2026The Confident Liar: Diagnosing Multi-Agent Debate with Log-Probabilities and LLM-as-Judgecited by 1
  25. 2026Early-Token Confidence Predicts Reasoning Quality in Multi-Agent LLM Debatecited by 1
  26. 2026Sigma-Mem: An Online Reliability Memory for LLM-based Multi-Agent Systemscited by 1
  27. 2026Refute-or-Promote: An Adversarial Stage-Gated Multi-Agent Review Methodology for High-Precision LLM-Assisted Defect Discoverycited by 0
  28. 2026Fair Agents: Balancing Multistakeholder Alignment in Multi-Agent Personalization Systemscited by 0
  29. 2026MACAA: Belief-Revision Multi-Agent Reasoning for Code Authorship Verificationcited by 0
  30. 2026Beyond Inefficiency: Systemic Costs of Incivility in Multi-Agent Monte Carlo Simulationscited by 0
  31. 2026Automated Detection and Classification of Delusion-related Content in Naturalistic Audio Diaries Using Multi-Agent Language Modelscited by 0
  32. 2026When Cloud Agents Meet Device Agents: Lessons from Hybrid Multi-Agent Systemscited by 0
  33. 2026Design and Evaluation of Multi-Agent AI Oracle Systems for Prediction Market Resolutioncited by 0
  34. 2026Durable Evaluation Framework: Adversarial Arbitration for Sycophancy Reduction in Large Language Modelscited by 0
  35. 2026Confidence Composition for Multiagent Language Model Systemscited by 0
  36. 2026Tandem Reinforcement Learning with Verifiable Rewardscited by 0
  37. 2026BOUNDARY_SYNC: Measuring Communication-Induced Representational Coupling in Multi-Agent LLM Systemscited by 0
  38. 2026TRACE: An Operational Reasoning Schema for Auditable Agentic Commitmentscited by 0

50 most cited of 63 All 63 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Can LLM Agents Really Debate? A Controlled Study of Multi-Agent Debate in Logical Reasoning
  2. 2025Interaction Protocol Shapes Moral Judgment in Multi-Agent Debate
  3. 2025Revisiting Multi-Agent Debate as Test-Time Scaling: A Systematic Study of Conditional Effectiveness
  4. 2025Stay Focused: Problem Drift in Multi-Agent Debate
  5. 2026When Debate Helps: Proposal Supply and Verification-Aware Readout in Multi-Agent Reasoning
  6. 2026Mixture of Debaters: Learn to Debate at Architectural Level in Multi-Agent Reasoning
  7. 2025When Disagreements Elicit Robustness: Investigating Self-Repair Capabilities under LLM Multi-Agent Disagreements
  8. 2026Multi-Agent Debate Strategies: Survey, Taxonomy, and Challenges