mlsanityData to 7 Oct 2026

Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models?

arXiv 2508.17536 ↗First posted 24 Aug 2025cs.CL cs.MAcited by 74

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Multi-Agent Debate~(MAD) has emerged as a promising paradigm for improving the performance of large language models through collaborative reasoning. Despite recent advances, the key factors driving MAD's effectiveness remain unclear. In this work, we disentangle MAD into two key components--Majority Voting and inter-agent Debate--and assess their respective contributions. Through extensive experiments across seven NLP benchmarks, we find that Majority Voting alone accounts for most of the performance gains typically attributed to MAD. To explain this, we propose a theoretical framework that models debate as a stochastic process. We prove that it induces a martingale over agents' belief trajectories, implying that debate alone does not improve expected correctness. Guided by these insights, we demonstrate that targeted interventions, by biasing the belief update toward correction, can meaningfully enhance debate effectiveness. Overall, our findings suggest that while MAD has potential, simple ensembling methods remain strong and more reliable alternatives in many practical settings. Code is released in https://github.com/deeplearning-wisc/debate-or-vote.

Key facts. Cited by 74 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 37 of its 48 references that are in the map. First posted on arXiv 24 Aug 2025.

Builds on

37 of 48 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024GroupDebate: Enhancing the Efficiency of Multi-Agent Debate Using Group Discussion21 Sep
  2. 2024Improving Multi-Agent Debate with Sparse Communication Topology17 Jun
  3. 2024Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?28 Feb
  4. 2023Let Models Speak Ciphers: Multiagent Debate through Embeddings10 Oct
  5. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs22 Sep
  6. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate30 May

Other references · 31

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2024Qwen2.5 Technical Reportcited by 4,713
  5. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  6. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  7. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  8. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  9. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  10. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  11. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
  12. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 752
19 more
  1. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
  2. 2020Aligning AI With Shared Human Valuescited by 706
  3. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 615
  4. 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 580
  5. 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
  6. 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 454
  7. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 424
  8. 2023MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoningcited by 311
  9. 2023Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaborationcited by 246
  10. 2024Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systemscited by 129
  11. 2023PRD: Peer Rank and Discussion Improve Large Language Model based Evaluationscited by 114
  12. 2023Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMscited by 103
  13. 2023Examining Inter-Consistency of Large Language Models Collaboration: An In-depth Analysis via Debatecited by 94
  14. 2025Beyond Self-Talk: A Communication-Centric Survey of LLM-Based Multi-Agent Systemscited by 58
  15. 2025Voting or Consensus? Decision-Making in Multi-Agent Debatecited by 55
  16. 2024Optima: Optimizing Effectiveness and Efficiency for LLM-Based Multi-Agent Systemcited by 40
  17. 2024Unveiling Selection Biases: Exploring Order and Token Sensitivity in Large Language Modelscited by 20
  18. 2024Mitigating Selection Bias with Node Pruning and Auxiliary Optionscited by 12
  19. 2024Exploring and Controlling Diversity in LLM-Agent Conversationcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 7; 2026: 67.

74 papers.

Sort:

By area:

  1. 2025Can LLM Agents Really Debate? A Controlled Study of Multi-Agent Debate in Logical Reasoningcited by 27
  2. 2025Beyond Majority Voting: LLM Aggregation by Leveraging Higher-Order Informationcited by 25
  3. 2026Understanding Agent Scaling in LLM-Based Multi-Agent Systems via Diversitycited by 25
  4. 2026Demystifying Multi-Agent Debate: The Role of Confidence and Diversitycited by 17
  5. 2026Multi-Agent Teams Hold Experts Backcited by 16
  6. 2026Uncertainty Quantification in LLM Agents: Foundations, Emerging Challenges, and Opportunitiescited by 15
  7. 2025AgenticSciML: Collaborative Multi-Agent Systems for Emergent Discovery in Scientific Machine Learningcited by 13
  8. 2026MAD-OPD: Breaking the Ceiling in On-Policy Distillation via Multi-Agent Debatecited by 12
  9. 2026Multi-Agent Debate with Memory Maskingcited by 8
  10. 2026Revision or Re-Solving? Decomposing Second-Pass Gains in Multi-LLM Pipelinescited by 7
  11. 2026ModeX: Evaluator-Free Best-of-N Selection for Open-Ended Generationcited by 5
  12. 2026DynaDebate: Breaking Homogeneity in Multi-Agent Debate with Dynamic Path Generationcited by 5
38 more
  1. 2026Toward Reliable Evaluation of LLM-Based Financial Multi-Agent Systems: Taxonomy, Coordination Primacy, and Cost Awarenesscited by 5
  2. 2026Hear Both Sides: Efficient Multi-Agent Debate via Diversity-Aware Message Retentioncited by 4
  3. 2026Scaling Discovery through Test-Time Communicationcited by 4
  4. 2025Many-to-One Adversarial Consensus: Exposing Multi-Agent Collusion Risks in AI-Based Healthcarecited by 3
  5. 2026TopoDIM: One-shot Topology Generation of Diverse Interaction Modes for Multi-Agent Systemscited by 3
  6. 2026PRISM: Pushing the Frontier of Deep Think via Process Reward Model-Guided Inferencecited by 3
  7. 2026Beyond Consensus: Trace-Level Synthesis in Mixture of Agentscited by 3
  8. 2026ATLAS: Agentic Test-time Learning-to-Allocate Scalingcited by 3
  9. 2026The Ringelmann Effect in Multi-Agent LLM Systems: A Scaling Law for Effective Team Sizecited by 3
  10. 2025Multi-Agent Collaborative Intelligence: Dual-Dial Control for Reliable LLM Reasoningcited by 2
  11. 2025Interaction Protocol Shapes Moral Judgment in Multi-Agent Debatecited by 2
  12. 2026Breaking the Martingale Curse: Multi-Agent Debate via Asymmetric Cognitive Potential Energycited by 2
  13. 2026EMS: Multi-Agent Voting via Efficient Majority-then-Stoppingcited by 2
  14. 2026Beyond Majority Voting: Efficient Best-Of-N with Radial Consensus Scorecited by 2
  15. 2026Belief Engine: Configurable and Inspectable Stance Dynamics in Multi-Agent LLM Deliberationcited by 2
  16. 2026Emergence of Biased Consensus in Multi-Agent LLM Debatescited by 2
  17. 2026XBridge: Entity-Grounded Latent Bridge for Heterogeneous LLM Communicationcited by 2
  18. 2026Flag Game: A Toy Model for Mechanistic Swarm Interpretabilitycited by 2
  19. 2025RevoNAD: Reflective Evolutionary Exploration for Neural Architecture Designcited by 1
  20. 2026Doc2AHP: Inferring Structured Multi-Criteria Decision Models via Semantic Trees with LLMscited by 1
  21. 2026Learning from Self-Debate: Preparing Reasoning Models for Multi-Agent Debatecited by 1
  22. 2026UrbanAlign: Post-hoc Semantic Calibration for VLM-Human Preference Alignmentcited by 1
  23. 2026Mitigating Hallucination on Hallucination in RAG via Ensemble Votingcited by 1
  24. 2026Weak-Link Optimization for Multi-Agent Reasoning and Collaborationcited by 1
  25. 2026What Do Agents Communicate? Characterizing Information Exchange in Multi-Agent Systemscited by 1
  26. 2026From Argument Components to Graphs: A Multi-Agent Debate with Confidence Gating for Argument Relationscited by 1
  27. 2026Budgeted Act-or-Defer Multi-Agent LLM Deliberation with Local Reliability Boundscited by 1
  28. 2026Sigma-Mem: An Online Reliability Memory for LLM-based Multi-Agent Systemscited by 1
  29. 2026Candidate supply and answer selection shape the value of LLM judging in multi-agent systemscited by 1
  30. 2026From Review to Authorization: Key-Isolated Threshold Signing for LLM Agentscited by 1
  31. 2026Self-Organizing Agent Teams Learn to Reason Togethercited by 1
  32. 2026ReCCur: A Recursive Corner-Case Curation Framework for Robust Vision-Language Understanding in Open and Edge Scenarioscited by 0
  33. 2026Secure and Energy-Efficient Wireless Agentic AI Networkscited by 0
  34. 2026The impact of multi-agent debate protocols on debate quality: a controlled case studycited by 0
  35. 2026CARE: Privacy-Compliant Agentic Reasoning with Evidence Discordancecited by 0
  36. 2026Learning in Blocks: A Multi Agent Debate Assisted Personalized Adaptive Learning Framework for Language Learningcited by 0
  37. 2026Multi-agent decision making: A Blackwell's informativeness approachcited by 0
  38. 2026When Independent Sampling Outperforms Agentic Reasoningcited by 0

50 most cited of 74 All 74 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Free-MAD: Consensus-Free Multi-Agent Debate
  2. 2026Multi-Agent Debate Strategies: Survey, Taxonomy, and Challenges
  3. 2025S2-MAD: Breaking the Token Barrier to Enhance Multi-Agent Debate Efficiency
  4. 2025When and Why Does Multi-Agent Debate Fail and Does It Really Underperform?
  5. 2025Self-Improvement of Language Models by Post-Training on Multi-Agent Debate
  6. 2025Stop Overvaluing Multi-Agent Debate -- We Must Rethink Evaluation and Embrace Model Heterogeneity
  7. 2025Enhancing Multi-Agent Debate System Performance via Confidence Expression
  8. 2026Remember and Reweight: Enhancing Multi-Agent Debate with Experience Memory and Confidence Estimation