# When Does Combining Language Models Help? A Co-Failure Ceiling on Routing, Voting, and Mixture-of-Agents Across 67 Frontier Models

arXiv: https://arxiv.org/abs/2606.27288 · HTML: https://mlsanity.com/paper/2606.27288/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 17 of its 38 references that are in the map.
- First posted on arXiv 25 Jun 2026.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 17 of 17)

- [FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance](https://mlsanity.com/paper/2305.05176/) (2023)
- [Selective Classification for Deep Neural Networks](https://mlsanity.com/paper/1705.08500/) (2017)
- [RouteLLM: Learning to Route LLMs with Preference Data](https://mlsanity.com/paper/2406.18665/) (2024)
- [LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion](https://mlsanity.com/paper/2306.02561/) (2023)
- [Mixture-of-Agents Enhances Large Language Model Capabilities](https://mlsanity.com/paper/2406.04692/) (2024)
- [Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing](https://mlsanity.com/paper/2404.14618/) (2024)
- [RouterBench: A Benchmark for Multi-LLM Routing System](https://mlsanity.com/paper/2403.12031/) (2024)
- [More Agents Is All You Need](https://mlsanity.com/paper/2402.05120/) (2024)
- [AutoMix: Automatically Mixing Language Models](https://mlsanity.com/paper/2310.12963/) (2023)
- [Algorithmic Monoculture and Social Welfare](https://mlsanity.com/paper/2101.05853/) (2021)
- [A Unified Approach to Routing and Cascading for LLMs](https://mlsanity.com/paper/2410.10347/) (2024)
- [A Unified Theory of Diversity in Ensemble Learning](https://mlsanity.com/paper/2301.03962/) (2023)
- [When Does Confidence-Based Cascade Deferral Suffice?](https://mlsanity.com/paper/2307.02764/) (2023)
- [Rethinking Mixture-of-Agents: Is Mixing Different Large Language Models Beneficial?](https://mlsanity.com/paper/2502.00674/) (2025)
- [Correlated Errors in Large Language Models](https://mlsanity.com/paper/2506.07962/) (2025)
- [Cost-of-Pass: An Economic Framework for Evaluating Language Models](https://mlsanity.com/paper/2504.13359/) (2025)
- [Don't Always Pick the Highest-Performing Model: An Information Theoretic View of LLM Ensemble Selection](https://mlsanity.com/paper/2602.08003/) (2026)

## Built on by (most cited first, 13 of 13)

- [Are Diversity Metrics Measuring Diversity? A Capability-Controlled Audit of Majority-Vote Gain in LLM Ensembles](https://mlsanity.com/paper/2607.20768/) (2026)
- [What Does a Routing Oracle Measure Under Stochastic Decoding? Coupling, Scorer Choice, and Single-Commit Ceilings](https://mlsanity.com/paper/2607.03436/) (2026)
- [Partially Correlated Verifier Cascades in LLM Harnesses: Concave Log-Odds, Polynomial Reliability, and Blind-Spot Ceilings](https://mlsanity.com/paper/2607.13918/) (2026)
- [Evaluator Ensembles Under Reward Hacking: Covariance Geometry and Finite-Search Guarantees](https://mlsanity.com/paper/2608.08002/) (2026)
- [Decorrelation Is Not Complementarity: Skill, Not Lineage, Governs Trusted-Monitor Ensembles](https://mlsanity.com/paper/2608.16190/) (2026)
- [What Does Multi-Agent LLM Debate Actually Change? A Layered Analysis of Disagreement and Answer Quality](https://mlsanity.com/paper/2609.08016/) (2026)
- [UnitBoost: Managing Compound LLM Systems with a Merge Operator, Not a Model](https://mlsanity.com/paper/2609.09815/) (2026)
- [Oracle Gaps in Reliability Coverage: Sampling Noise or Policy Specialization?](https://mlsanity.com/paper/2609.32996/) (2026)
- [On the Limits of Metacognitive Monitoring in LLMs](https://mlsanity.com/paper/2609.34864/) (2026)
- [False Floors: LLM Safety Routing Evaluations Break Under Distribution Shift](https://mlsanity.com/paper/2610.01535/) (2026)
- [CopRoute: Dependence-Aware Conformal Routing for Language Models](https://mlsanity.com/paper/2610.13276/) (2026)
- [Handoff Curves: Initial Selection, Later Handoff, and Supervision Targets in Multi-turn LLM Routing](https://mlsanity.com/paper/2610.15993/) (2026)
- [Where Do LLM Ensemble Gains Come From? Decomposing Candidate Value, Diversity, and Aggregation](https://mlsanity.com/paper/2610.18786/) (2026)

## Abstract (arXiv)

Multi-model LLM systems such as routing, voting, cascades, fusion, and mixture-of-agents are used to beat single-model accuracy. We show that their gain is capped by a quantity the field rarely reports. For any policy whose output is one member model answer, accuracy cannot exceed one minus beta, where beta is the rate at which every model is wrong on the same query. In contrast, the usual diagnostic, average pairwise error correlation rho, cannot identify beta: error laws with identical marginals and pairwise correlations can have different all-wrong rates. A Clopper-Pearson bound on beta gives a finite-sample certificate on the largest gain any router, vote, or cascade could deliver before training a router. Across 67 models from 21 providers, a tetrachoric-calibrated single-factor model still underprices the all-wrong tail: on open-ended mathematics, observed beta is 0.052 versus 0.023 under the full 67-model Gaussian copula, about 2.5 times underpricing, with 90 percent CI 1.7 to 3.4 and k equals 17. The effect recurs on execution-graded code, where beta is 0.079. Re-asking the same GPQA-Diamond questions in free-response rather than multiple-choice form reopens the tail, with beta 0.127 and a five-judge panel with kappa 0.73 to 0.92, locating co-failure in answer format rather than subject. At matched quality, low-rho heterogeneous ensembles beat high-rho Self-MoA, but on checkable tasks in our pool, combining models rarely beats the single best model without a strong query-level routing signal. Gains come from models failing on different questions, not from adding more models.