Beyond Symmetric Agents: Cognitive Diversity and Multi-Agent Debate in Small Language Models
On the map: LLM agents and tool use → Social simulation and multi-agent debate
Abstract (arXiv)
Multi-agent debate (MAD) reportedly improves reasoning and factuality over single-model inference, but prior work treats agents as symmetric peers, leaving open what drives the gains. We test the hypothesis that cognitive diversity among agents is the driver, in the setting where the question is still measurable: small open-weight models with benchmark headroom. Across 23 models from eleven vendor families, five tasks, and 5,500+ debate and control runs, we vary diversity along three axes - personas, sampling temperature, and model identity - pairing every debate configuration with a generation-budget-matched majority-vote control. The hypothesis is rejected on every axis. Debate beats single-agent inference (3--7 points where tasks have headroom) but at matched budget conditions it ties or even loses to self-consistency sampling at 1.6× the wall-clock and 3.4× the token cost. Persona prompting reduces accuracy and a dose-response experiment over each model's full combinatorial persona space shows the cost is a persona tax, not a diversity tax: redundant personas hurt most, while maximally-diverse teams recover part of the loss. Furthermore, mixed-model teams lose to majority votes over their own rosters, with accuracy tracking member capability rather than heterogeneity, and nearly all of debate's benefit comes from the first exchange of answers. We further identify a pervasive measurement hazard in which debate transcripts silently overflow serving context windows, whose correction alone moves our debate-versus-sampling comparison from -1.8 points to parity. Our results recast reported MAD gains as an ensemble-sampling effect and provide the budget-matched, contamination-checked baseline bar that future debate mechanisms should be required to clear.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 27 of its 36 references that are in the map. First posted on arXiv 26 Sep 2026.
Builds on
27 of 36 references.
Closest ancestors · 6
Same direction, newest first
- 2026DynaDebate: Breaking Homogeneity in Multi-Agent Debate with Dynamic Path Generation9 Jan
- 2025Can LLM Agents Really Debate? A Controlled Study of Multi-Agent Debate in Logical Reasoning11 Nov
- 2025Talk Isn't Always Cheap: Understanding Failure Modes in Multi-Agent Debate5 Sep
- 2025Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models?24 Aug
- 2023Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMs29 Nov
- 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate30 May
Other references · 21
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
- 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
- 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
- 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,463
- 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 742
- 2018Universal Transformerscited by 647
- 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 415
- 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMscited by 309
9 more
- 2024Debating with More Persuasive LLMs Leads to More Truthful Answerscited by 266
- 2025Reasoning with Latent Thoughts: On the Power of Looped Transformerscited by 240
- 2025Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computationcited by 133
- 2023Let's Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMscited by 115
- 2024Escape Sky-high Cost: Early-stopping Self-Consistency for Multi-step Reasoningcited by 108
- 2024On scalable oversight with weak LLMs judging strong LLMscited by 99
- 2024Make Every Penny Count: Difficulty-Adaptive Self-Consistency for Cost-Efficient Reasoningcited by 63
- 2025Ranked Voting based Self-Consistency of Large Language Modelscited by 19
- 2026Reliability-Aware Adaptive Self-Consistency for Efficient Sampling in LLM Reasoningcited by 11
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Diversity of Thought Elicits Stronger Reasoning Capabilities in Multi-Agent Debate Frameworks
- 2026Demystifying Multi-Agent Debate: The Role of Confidence and Diversity
- 2026Creative Generation via Multi-Agent Debate: Does Debate Suppress Diversity?
- 2025Revisiting Multi-Agent Debate as Test-Time Scaling: A Systematic Study of Conditional Effectiveness
- 2026Mixture of Debaters: Learn to Debate at Architectural Level in Multi-Agent Reasoning
- 2025The Social Laboratory: A Psychometric Framework for Multi-Agent LLM Evaluation
- 2026When Order Matters: First-Speaker Bias and Mitigation through Personality in Sequential Multi-Agent Debate
- 2025Stop Overvaluing Multi-Agent Debate -- We Must Rethink Evaluation and Embrace Model Heterogeneity