Measuring Collapse and Correction in Homogeneous-Panel LLM Debate
On the map: LLM agents and tool use → Social simulation and multi-agent debate
Abstract (arXiv)
Multi-agent large language model (LLM) debate is often evaluated by whether final answers improve, but movement is not necessarily improvement: the same discussion can rescue an initially wrong majority or destroy an initially correct one. Standard final-accuracy evaluations conflate these opposing mechanisms. We introduce an auditable protocol for homogeneous debate on multiple-choice questions (MCQs) that records each run as a transition ledger over collapse, correction, onset, and signed intervention utility. On 6,925 MMLU-Pro debates, the protocol identifies 253 collapses and a parallel correction ledger that changes how interventions should be judged. Replay experiments reveal the central tradeoff: a leave-one-model-out probe-gated freeze prevents 29 collapses but loses 108 corrections under equal weights, so collapse prevention alone can recommend the wrong policy. A compact pre-debate 8-probe screen is a triage signal: its unadjusted family-level association with conditional-collapse risk is high (G=7, Spearman rho=0.893, exact two-sided p=0.0123), but initial-majority accuracy is a close comparator (rho=0.821; family partial rho=0.767, p=0.0877), so we do not treat it as calibrated or capability-adjusted prediction. Round-level traces localize many collapses to the first debate round, where early disagreement can precede both harmful cascades and useful recovery. We release replayable schemas, coders, audits, cost cards, and zero-API rebuild scripts so future model-scaffold rows can be compared under the same denominators and signed utility ledger.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 28 of its 32 references that are in the map. First posted on arXiv 28 Sep 2026.
Builds on
28 of 32 references.
Closest ancestors · 7
Same direction, newest first
- 2026Too Polite to Disagree: Understanding Sycophancy Propagation in Multi-Agent Systems3 Apr
- 2026The Value of Variance: Mitigating Debate Collapse in Multi-Agent Systems via Uncertainty-Driven Policy Optimization6 Feb
- 2026Demystifying Multi-Agent Debate: The Role of Confidence and Diversity9 Jan
- 2025iMAD: Intelligent Multi-Agent Debate for Efficient and Accurate LLM Inference14 Nov
- 2025Can LLM Agents Really Debate? A Controlled Study of Multi-Agent Debate in Logical Reasoning11 Nov
- 2025Talk Isn't Always Cheap: Understanding Failure Modes in Multi-Agent Debate5 Sep
- 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate30 May
Other references · 21
- 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
- 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
- 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
- 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
- 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
- 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
- 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
- 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 742
- 2023Simple synthetic data reduces sycophancy in large language modelscited by 187
- 2023LM vs LM: Detecting Factual Errors via Cross Examinationcited by 156
- 2025Confidence Improves Self-Consistency in LLMscited by 113
9 more
- 2023Examining Inter-Consistency of Large Language Models Collaboration: An In-depth Analysis via Debatecited by 97
- 2025Debate Only When Necessary: Adaptive Multiagent Collaboration for Efficient LLM Reasoningcited by 27
- 2025Scaling Laws For Scalable Oversightcited by 23
- 2025The Sequential Edge: Inverse-Entropy Voting Beats Parallel Self-Consistency at Matched Computecited by 11
- 2025When Two LLMs Debate, Both Think They'll Wincited by 10
- 2026LLMs Know They're Wrong and Agree Anyway: The Shared Sycophancy-Lying Circuitcited by 10
- 2025Disentangling the Drivers of LLM Social Conformity: An Uncertainty-Moderated Dual-Process Mechanismcited by 6
- 2025CGES: Confidence-Guided Early Stopping for Efficient and Accurate Self-Consistencycited by 6
- 2026Knowledge Divergence and the Value of Debate for Scalable Oversightcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026What Does Multi-Agent LLM Debate Actually Change? A Layered Analysis of Disagreement and Answer Quality
- 2026The Cost of Consensus: Isolated Self-Correction Prevails Over Unguided Homogeneous Multi-Agent Debate
- 2023Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMs
- 2026Black-Box Auditing of Epistemic Reliability in Multi-Agent Debate Distillation
- 2026Not All Flips Are Conformity: Decomposing Stance Convergence in Multi-Agent LLM Debate
- 2026Statistical Scouting Finds Debate-Safe but Not Debate-Useful Cases: A Matched-Ceiling Study of Open-Weight LLM Reasoning Protocols
- 2025Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models?
- 2025DS@GT at Touch'e: Large Language Models for Retrieval-Augmented Debate