# Can LLM Agents Really Debate? A Controlled Study of Multi-Agent Debate in Logical Reasoning

arXiv: https://arxiv.org/abs/2511.07784 · HTML: https://mlsanity.com/paper/2511.07784/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 21 of its 36 references that are in the map.
- First posted on arXiv 11 Nov 2025.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 21 of 21)

- [ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate](https://mlsanity.com/paper/2308.07201/) (2023)
- [Debating with More Persuasive LLMs Leads to More Truthful Answers](https://mlsanity.com/paper/2402.06782/) (2024)
- [Improving Multi-Agent Debate with Sparse Communication Topology](https://mlsanity.com/paper/2406.11776/) (2024)
- [Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models?](https://mlsanity.com/paper/2508.17536/) (2025)
- [GroupDebate: Enhancing the Efficiency of Multi-Agent Debate Using Group Discussion](https://mlsanity.com/paper/2409.14051/) (2024)
- [Voting or Consensus? Decision-Making in Multi-Agent Debate](https://mlsanity.com/paper/2502.19130/) (2025)
- [Can LLMs Produce Faithful Explanations For Fact-checking? Towards Faithful Explainable Fact-Checking via Multi-Agent Debate](https://mlsanity.com/paper/2402.07401/) (2024)
- [Stop Overvaluing Multi-Agent Debate -- We Must Rethink Evaluation and Embrace Model Heterogeneity](https://mlsanity.com/paper/2502.08788/) (2025)
- [X-MAS: Towards Building Multi-Agent Systems with Heterogeneous LLMs](https://mlsanity.com/paper/2505.16997/) (2025)
- [Debate Only When Necessary: Adaptive Multiagent Collaboration for Efficient LLM Reasoning](https://mlsanity.com/paper/2504.05047/) (2025)
- [Revisiting Multi-Agent Debate as Test-Time Scaling: A Systematic Study of Conditional Effectiveness](https://mlsanity.com/paper/2505.22960/) (2025)
- [Enhancing Multi-Agent Debate System Performance via Confidence Expression](https://mlsanity.com/paper/2509.14034/) (2025)
- [Interpreting and Mitigating Hallucination in MLLMs through Multi-agent Debate](https://mlsanity.com/paper/2407.20505/) (2024)
- [Diversity of Thought Elicits Stronger Reasoning Capabilities in Multi-Agent Debate Frameworks](https://mlsanity.com/paper/2410.12853/) (2024)
- [When Persuasion Overrides Truth in Multi-Agent LLM Debates: Introducing a Confidence-Weighted Persuasion Override Rate (CW-POR)](https://mlsanity.com/paper/2504.00374/) (2025)
- [Enhancing LLM Performance Through Debate: An Empirical Study on Multi-Agent Debate for Coding Tasks](https://mlsanity.com/paper/2503.12029/) (2025)
- [DebFlow: Automating Agent Creation via Agent Debate](https://mlsanity.com/paper/2503.23781/) (2025)
- [Tiered Agentic Oversight: A Hierarchical Multi-Agent System for Healthcare Safety](https://mlsanity.com/paper/2506.12482/) (2025)
- [Debating Truth: Debate-driven Claim Verification with Multiple Large Language Model Agents](https://mlsanity.com/paper/2507.19090/) (2025)
- [From Belief Entrenchment to Robust Reasoning in LLM Agents](https://mlsanity.com/paper/2503.16814/) (2025)
- [Revealing Political Bias in LLMs through Structured Multi-Agent Debate](https://mlsanity.com/paper/2506.11825/) (2025)

## Built on by (most cited first, 29 of 29)

- [Demystifying Multi-Agent Debate: The Role of Confidence and Diversity](https://mlsanity.com/paper/2601.19921/) (2026)
- [Scaling Discovery through Test-Time Communication](https://mlsanity.com/paper/2609.21032/) (2026)
- [Minority Sentinel: When to Overturn Majority Voting in Multi-Agent LLM Debates](https://mlsanity.com/paper/2606.29270/) (2026)
- [More Rounds, More Noise: Why Multi-Turn Review Fails to Improve Cross-Context Verification](https://mlsanity.com/paper/2603.16244/) (2026)
- [Hear Both Sides: Efficient Multi-Agent Debate via Diversity-Aware Message Retention](https://mlsanity.com/paper/2603.20640/) (2026)
- [Courtroom-Style Multi-Agent Debate with Progressive RAG and Role-Switching for Controversial Claim Verification](https://mlsanity.com/paper/2603.28488/) (2026)
- [Multi-Agent Dialectical Refinement for Enhanced Argument Classification](https://mlsanity.com/paper/2603.27451/) (2026)
- [AgentCity: Constitutional Governance for Autonomous Agent Economies via Separation of Power](https://mlsanity.com/paper/2604.07007/) (2026)
- [The Ringelmann Effect in Multi-Agent LLM Systems: A Scaling Law for Effective Team Size](https://mlsanity.com/paper/2606.02646/) (2026)
- [Precise but Uncoupled: Reviewer Precision Does Not Guarantee Critique Uptake in Multi-Agent Math Reasoning](https://mlsanity.com/paper/2607.15388/) (2026)
- [The Deliberative Deficit: An Empirical Critique of LLMs in Democratic Discourse](https://mlsanity.com/paper/2608.10186/) (2026)
- [From Argument Components to Graphs: A Multi-Agent Debate with Confidence Gating for Argument Relations](https://mlsanity.com/paper/2606.16047/) (2026)
- [What LLM Agents Say When No One Is Watching: Social Structure and Latent Objective Emergence in Multi-Agent Debates](https://mlsanity.com/paper/2607.02507/) (2026)
- [One Axis, No Brake: Self-Knowledge Limits the Filtering of Harmful Peer Conformity in LLMs](https://mlsanity.com/paper/2609.18998/) (2026)
- [BARRED: Synthetic Training of Custom Policy Guardrails via Asymmetric Debate](https://mlsanity.com/paper/2604.25203/) (2026)
- [When Helping Hurts and How to Fix It: Multi-Agent Debate for Data Cleaning](https://mlsanity.com/paper/2606.02866/) (2026)
- [MoCA-Agent: A Market-of-Claims Code Agent for Financial and Numerical Reasoning](https://mlsanity.com/paper/2606.11537/) (2026)
- [Confidence Composition for Multiagent Language Model Systems](https://mlsanity.com/paper/2606.13591/) (2026)
- [Automatic Hard Example Synthesis with Multi-Level Agentic Data Curation](https://mlsanity.com/paper/2607.14256/) (2026)
- [Do LLM Debates Repeat Arguments Differently Across Languages?](https://mlsanity.com/paper/2607.23442/) (2026)
- [MADE: Belief-Driven Dual-Agent Coordination for Autonomous Model Deployment](https://mlsanity.com/paper/2608.01189/) (2026)
- [Process-Constituted Intelligence: A Shared Criterion for Humans and Machines](https://mlsanity.com/paper/2608.16213/) (2026)
- [Unanimity Without Persuasion: A Single Round of Debate Erases the Disagreement That Verification Needs](https://mlsanity.com/paper/2609.26145/) (2026)
- [What Confidence Routing Is Actually Doing: Auditing Routing, Calibration, and Commitment in Multi-Agent Deliberation](https://mlsanity.com/paper/2609.27822/) (2026)
- [Black-Box Auditing of Epistemic Reliability in Multi-Agent Debate Distillation](https://mlsanity.com/paper/2609.32361/) (2026)
- [Measuring Collapse and Correction in Homogeneous-Panel LLM Debate](https://mlsanity.com/paper/2609.35279/) (2026)
- [Beyond Symmetric Agents: Cognitive Diversity and Multi-Agent Debate in Small Language Models](https://mlsanity.com/paper/2609.35875/) (2026)
- [Multi-agent discussion gains less when dissent is withheld](https://mlsanity.com/paper/2609.38324/) (2026)
- [Peer Influence across Heterogeneous AI Models](https://mlsanity.com/paper/2610.03095/) (2026)

## Abstract (arXiv)

Multi-agent debate (MAD) has recently emerged as a promising framework for improving the reasoning performance of large language models (LLMs). Yet, whether LLM agents can genuinely engage in deliberative reasoning, beyond simple ensembling or majority voting, remains unclear. We address this question through a controlled study using the Knight--Knave--Spy logic puzzle, which enables precise, step-wise evaluation of debate outcomes and processes under verifiable ground truth. We systematically set up six structural and cognitive factors, including agent team size, composition, confidence visibility, debate order, debate depth, and task difficulty, to disentangle their respective effects on collective reasoning. Our results show that intrinsic reasoning strength and group diversity are the dominant drivers of debate success, while structural parameters such as order or confidence visibility offer limited gains. Beyond outcomes, process-level analyses identify key behavioral patterns: majority pressure suppresses independent correction, effective teams overturn incorrect consensus, and rational, validity-aligned reasoning most strongly predicts improvement. These findings provide valuable insights into how and why LLM debates succeed or fail, offering guidance for designing interpretable and truth-seeking multi-agent reasoning systems.