mlsanityData to 7 Oct 2026

Are Diversity Metrics Measuring Diversity? A Capability-Controlled Audit of Majority-Vote Gain in LLM Ensembles

arXiv 2607.20768 ↗First posted 22 Jul 2026cs.CL cs.AI cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Majority voting over LLMs is widely assumed to benefit from diversity, and diversity measures are used to choose which models to combine. We ask whether five such measures track diversity or mainly re-express capability, auditing them as predictors of majority-vote gain over the best member across 31,900 subsets of 30 LLMs on MMLU-Pro (29 on TruthfulQA) under explicit capability controls. Three findings emerge. First, latent complementarity is ubiquitous: oracle gain is positive in 100% of subsets, yet simple voting beats the strongest member in only 9.98% of all canonical size-3 subsets (18.71% with held-out best selection); the pooled size-2-4 rate is 1.27%, partly reflecting deterministic even-size voting behavior. Second, a joint-correctness proxy (strict diversity) is nearly collinear with one minus mean accuracy (size-3 Spearman rho = +0.991 / +0.988); raw diversity-gain associations are strongly capability-entangled and, with one exception, unstable under control. Third, three linear contingency-table statistics are algebraically non-separable; after capability control, the empirically stable remainder is a modest residual pairwise co-failure association in which more shared error corresponds to lower gain. This direction is robust, but its magnitude is configuration-dependent. Joint rawspace linear regressions treating strict diversity, disagreement, and double-fault as independent predictors are rank-deficient by construction.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 19 of its 35 references that are in the map. First posted on arXiv 22 Jul 2026.

Builds on

19 of 35 references.

References · 19

  1. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  2. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  3. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  4. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
  5. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 550
  6. 2024Mixture-of-Agents Enhances Large Language Model Capabilitiescited by 420
  7. 2025Harnessing Multiple Large Language Models: A Survey on LLM Ensemblecited by 86
  8. 2023A Unified Theory of Diversity in Ensemble Learningcited by 71
  9. 2025Great Models Think Alike and this Undermines AI Oversightcited by 53
  10. 2025Rethinking Mixture-of-Agents: Is Mixing Different Large Language Models Beneficial?cited by 52
  11. 2023Getting MoRE out of Mixture of Language Model Reasoning Expertscited by 44
  12. 2024LLM-TOPLA: Efficient LLM Ensemble by Maximising Diversitycited by 42
7 more
  1. 2025Beyond Majority Voting: LLM Aggregation by Leveraging Higher-Order Informationcited by 25
  2. 2026Don't Always Pick the Highest-Performing Model: An Information Theoretic View of LLM Ensemble Selectioncited by 10
  3. 2026When Does Combining Language Models Help? A Co-Failure Ceiling on Routing, Voting, and Mixture-of-Agents Across 67 Frontier Modelscited by 10
  4. 2024Examining Independence in Ensemble Sentiment Analysis: A Study on the Limits of Large Language Models Using the Condorcet Jury Theoremcited by 4
  5. 2025DFPE: A Diverse Fingerprint Ensemble for Enhancing LLM Performancecited by 4
  6. 2026Quantifying Diversity of Thought: A Predictive Law of Weighted LLM Ensemble Liftcited by 4
  7. 2026LLM-as-a-Judge Scores Are Unreliable Optimization Signals in Closed-Loop Table Recognitioncited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026Sixteen models, fewer than two voices: measuring ensemble dispersion where no answer is uniquely correctcited by 0
  2. 2026Decorrelation Is Not Complementarity: Skill, Not Lineage, Governs Trusted-Monitor Ensemblescited by 0
  3. 2026Decomposing Wrong-Consensus Agreement in LLM Self-Consistencycited by 0
  4. 2026A latent dimension of Condorcet's jury theorem for multiple AI adviserscited by 0
  5. 2026Feed the Panel Dimensions, Not Verdicts: Rubric-Decomposed Fusion of Vision-Language Aesthetic Judgescited by 0
  6. 2026Oracle Gaps in Reliability Coverage: Sampling Noise or Policy Specialization?cited by 0
  7. 2026RAIM: Robust Aggregation of Inexpensive Models for Hallucination Detectioncited by 0