# Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models?

arXiv: https://arxiv.org/abs/2508.17536 · HTML: https://mlsanity.com/paper/2508.17536/

## Key facts

- Cited by 81 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 37 of its 48 references that are in the map.
- First posted on arXiv 24 Aug 2025.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 30 of 37)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [Large Language Models Cannot Self-Correct Reasoning Yet](https://mlsanity.com/paper/2310.01798/) (2023)
- [Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate](https://mlsanity.com/paper/2305.19118/) (2023)
- [Large Language Model based Multi-Agents: A Survey of Progress and Challenges](https://mlsanity.com/paper/2402.01680/) (2024)
- [ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate](https://mlsanity.com/paper/2308.07201/) (2023)
- [Aligning AI With Shared Human Values](https://mlsanity.com/paper/2008.02275/) (2020)
- [RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback](https://mlsanity.com/paper/2309.00267/) (2023)
- [Why Do Multi-Agent LLM Systems Fail?](https://mlsanity.com/paper/2503.13657/) (2025)
- [AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors](https://mlsanity.com/paper/2308.10848/) (2023)
- [Large Language Models Are Not Robust Multiple Choice Selectors](https://mlsanity.com/paper/2309.03882/) (2023)
- [Multi-Agent Collaboration Mechanisms: A Survey of LLMs](https://mlsanity.com/paper/2501.06322/) (2025)
- [MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning](https://mlsanity.com/paper/2311.10537/) (2023)
- [ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs](https://mlsanity.com/paper/2309.13007/) (2023)
- [Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration](https://mlsanity.com/paper/2307.05300/) (2023)
- [Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?](https://mlsanity.com/paper/2402.18272/) (2024)
- [Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systems](https://mlsanity.com/paper/2410.02506/) (2024)
- [PRD: Peer Rank and Discussion Improve Large Language Model based Evaluations](https://mlsanity.com/paper/2307.02762/) (2023)
- [Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMs](https://mlsanity.com/paper/2311.17371/) (2023)
- [Improving Multi-Agent Debate with Sparse Communication Topology](https://mlsanity.com/paper/2406.11776/) (2024)
- [Examining Inter-Consistency of Large Language Models Collaboration: An In-depth Analysis via Debate](https://mlsanity.com/paper/2305.11595/) (2023)
- [GroupDebate: Enhancing the Efficiency of Multi-Agent Debate Using Group Discussion](https://mlsanity.com/paper/2409.14051/) (2024)

## Built on by (most cited first, 30 of 81)

- [Can LLM Agents Really Debate? A Controlled Study of Multi-Agent Debate in Logical Reasoning](https://mlsanity.com/paper/2511.07784/) (2025)
- [Understanding Agent Scaling in LLM-Based Multi-Agent Systems via Diversity](https://mlsanity.com/paper/2602.03794/) (2026)
- [Beyond Majority Voting: LLM Aggregation by Leveraging Higher-Order Information](https://mlsanity.com/paper/2510.01499/) (2025)
- [Multi-Agent Teams Hold Experts Back](https://mlsanity.com/paper/2602.01011/) (2026)
- [Uncertainty Quantification in LLM Agents: Foundations, Emerging Challenges, and Opportunities](https://mlsanity.com/paper/2602.05073/) (2026)
- [Demystifying Multi-Agent Debate: The Role of Confidence and Diversity](https://mlsanity.com/paper/2601.19921/) (2026)
- [MAD-OPD: Breaking the Ceiling in On-Policy Distillation via Multi-Agent Debate](https://mlsanity.com/paper/2605.01347/) (2026)
- [AgenticSciML: Collaborative Multi-Agent Systems for Emergent Discovery in Scientific Machine Learning](https://mlsanity.com/paper/2511.07262/) (2025)
- [Multi-Agent Debate with Memory Masking](https://mlsanity.com/paper/2603.20215/) (2026)
- [Revision or Re-Solving? Decomposing Second-Pass Gains in Multi-LLM Pipelines](https://mlsanity.com/paper/2604.01029/) (2026)
- [Scaling Discovery through Test-Time Communication](https://mlsanity.com/paper/2609.21032/) (2026)
- [ModeX: Evaluator-Free Best-of-N Selection for Open-Ended Generation](https://mlsanity.com/paper/2601.02535/) (2026)
- [DynaDebate: Breaking Homogeneity in Multi-Agent Debate with Dynamic Path Generation](https://mlsanity.com/paper/2601.05746/) (2026)
- [Toward Reliable Evaluation of LLM-Based Financial Multi-Agent Systems: Taxonomy, Coordination Primacy, and Cost Awareness](https://mlsanity.com/paper/2603.27539/) (2026)
- [Hear Both Sides: Efficient Multi-Agent Debate via Diversity-Aware Message Retention](https://mlsanity.com/paper/2603.20640/) (2026)
- [Beyond Consensus: Trace-Level Synthesis in Mixture of Agents](https://mlsanity.com/paper/2605.29116/) (2026)
- [ATLAS: Agentic Test-time Learning-to-Allocate Scaling](https://mlsanity.com/paper/2606.01667/) (2026)
- [Interaction Protocol Shapes Moral Judgment in Multi-Agent Debate](https://mlsanity.com/paper/2510.10002/) (2025)
- [Many-to-One Adversarial Consensus: Exposing Multi-Agent Collusion Risks in AI-Based Healthcare](https://mlsanity.com/paper/2512.03097/) (2025)
- [TopoDIM: One-shot Topology Generation of Diverse Interaction Modes for Multi-Agent Systems](https://mlsanity.com/paper/2601.10120/) (2026)
- [PRISM: Pushing the Frontier of Deep Think via Process Reward Model-Guided Inference](https://mlsanity.com/paper/2603.02479/) (2026)
- [Breaking the Martingale Curse: Multi-Agent Debate via Asymmetric Cognitive Potential Energy](https://mlsanity.com/paper/2603.06801/) (2026)
- [AgentCity: Constitutional Governance for Autonomous Agent Economies via Separation of Power](https://mlsanity.com/paper/2604.07007/) (2026)
- [The Ringelmann Effect in Multi-Agent LLM Systems: A Scaling Law for Effective Team Size](https://mlsanity.com/paper/2606.02646/) (2026)
- [Emergence of Biased Consensus in Multi-Agent LLM Debates](https://mlsanity.com/paper/2608.02827/) (2026)
- [XBridge: Entity-Grounded Latent Bridge for Heterogeneous LLM Communication](https://mlsanity.com/paper/2608.11676/) (2026)
- [Multi-Agent Collaborative Intelligence: Dual-Dial Control for Reliable LLM Reasoning](https://mlsanity.com/paper/2510.04488/) (2025)
- [EMS: Multi-Agent Voting via Efficient Majority-then-Stopping](https://mlsanity.com/paper/2604.02863/) (2026)
- [Beyond Majority Voting: Efficient Best-Of-N with Radial Consensus Score](https://mlsanity.com/paper/2604.12196/) (2026)
- [Belief Engine: Configurable and Inspectable Stance Dynamics in Multi-Agent LLM Deliberation](https://mlsanity.com/paper/2605.15343/) (2026)

## Abstract (arXiv)

Multi-Agent Debate~(MAD) has emerged as a promising paradigm for improving the performance of large language models through collaborative reasoning. Despite recent advances, the key factors driving MAD's effectiveness remain unclear. In this work, we disentangle MAD into two key components--Majority Voting and inter-agent Debate--and assess their respective contributions. Through extensive experiments across seven NLP benchmarks, we find that Majority Voting alone accounts for most of the performance gains typically attributed to MAD. To explain this, we propose a theoretical framework that models debate as a stochastic process. We prove that it induces a martingale over agents' belief trajectories, implying that debate alone does not improve expected correctness. Guided by these insights, we demonstrate that targeted interventions, by biasing the belief update toward correction, can meaningfully enhance debate effectiveness. Overall, our findings suggest that while MAD has potential, simple ensembling methods remain strong and more reliable alternatives in many practical settings. Code is released in https://github.com/deeplearning-wisc/debate-or-vote.