When Agents Disagree: The Selection Bottleneck in Multi-Agent LLM Pipelines
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
Multi-agent LLM pipelines produce contradictory evidence on whether team diversity improves output quality: heterogeneous Mixture-of-Agents teams outperform single models, yet homogeneous Self-MoA teams consistently win under synthesis-based aggregation. We propose a resolution by identifying the selection bottleneck -- a crossover threshold in aggregation quality that determines whether diversity helps or hurts. Under this model, we obtain a closed-form crossover threshold s^* (Proposition 1) that separates the regimes where diversity helps and hurts. In a targeted experiment spanning 42 tasks across 7 categories (N=210), a diverse team with judge-based selection achieves a win rate of 0.810 against a single-model baseline, while a homogeneous team scores 0.512 -- near chance (Glass's Delta = 2.07). Judge-based selection outperforms MoA-style synthesis by DeltaWR = +0.631 -- the synthesis approach is preferred over the baseline in zero of 42 tasks by the judge panel. A decoupled evaluation with independent judges confirms all directional findings (Spearman ρ = 0.90). Exploratory evidence suggests that including a weaker model improves performance while reducing cost (p < 10-4, not pre-registered). Our results suggest that selector quality may be a more impactful design lever than generator diversity in single-round generate-then-select pipelines.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 14 of its 26 references that are in the map. First posted on arXiv 20 Mar 2026.
Builds on
14 of 26 references.
Closest ancestors · 3
Same direction, newest first
- 2024More Agents Is All You Need3 Feb
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework1 Aug
- 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society31 Mar
Other references · 11
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
- 2020Learning to summarize from human feedbackcited by 2,562
- 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
- 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
- 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
- 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 651
- 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 550
- 2024Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Modelscited by 301
- 2024Large Language Models are Inconsistent and Biased Evaluatorscited by 146
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026CIPHER: A Decoupled Exploration-Selection Framework for Test-Time Scaling of Data Science Agentscited by 0
- 2026UnitBoost: Managing Compound LLM Systems with a Merge Operator, Not a Modelcited by 0
- 2026An Exact Generate - Transform Decomposition of Small-LLM Team Scaling Across Orchestration Architecturescited by 0
- 2026An Exact Generate - Transform Decomposition of Small-LLM Team Scaling Across Orchestration Architectures28 Sep 2026
- 2026UnitBoost: Managing Compound LLM Systems with a Merge Operator, Not a Model9 Sep 2026
- 2026CIPHER: A Decoupled Exploration-Selection Framework for Test-Time Scaling of Data Science Agents15 Jul 2026
Similar, not cited
Close in wording, no citation either way
- 2026Multi-agent Scaling Across Disjunctive and Compensatory Tasks
- 2026The Interaction Tax: When Communication Erases Diversity in Multi-Agent Teams
- 2026Understanding Agent Scaling in LLM-Based Multi-Agent Systems via Diversity
- 2026At Equal Inference Cost, Multi-Agent Structure Does Not Beat a Single Frozen Agent
- 2026Multi-Agent Teams Hold Experts Back
- 2026Beyond Static Leaderboards: Predictive Validity for the Evaluation of LLM Agents
- 2026When Agents Slow Down: Understanding LLM Agents' Test-Time Strategies via Elo-per-token Analysis
- 2026Diversity Collapse in Multi-Agent LLM Systems: Structural Coupling and Collective Failure in Open-Ended Idea Generation