Which Models Work Well Together? Measuring Heterogeneity for LLM Team Selection
Not on the map. Why?
Abstract (arXiv)
The performance ceiling of an LLM team is constrained not only by individual model capabilities, but also by inter-member error resonance and predictive differences. Although heterogeneous teaming is often observed to be effective in practice, existing approaches lack complementarity metrics that are computable, interpretable, and optimizable, leaving team composition to rely on heuristics. We propose a heterogeneity-driven team selection framework that performs offline profiling to characterize individual capability along with two complementary signals: one captures decorrelation in error patterns to reduce co-failures, while the other measures divergence in predictive behavior to capture strategy diversity. We formulate team selection as a standardized quality--complementarity combinatorial objective and apply an efficient greedy search to select a small team from a candidate pool. Experiments across multiple benchmarks demonstrate that our framework consistently outperforms quality-only baselines under controlled candidate pools and team sizes, establishing reusable selection principles for multi-LLM systems.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 60 of its 75 references that are in the map. First posted on arXiv 29 Sep 2026.
Builds on
60 of 75 references.
References · 60
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2025Qwen3 Technical Reportcited by 8,195
- 2020Measuring Massive Multitask Language Understandingcited by 7,768
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
- 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
- 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
- 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
- 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
- 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
38 more
- 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
- 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
- 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
- 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
- 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
- 2023ChatDev: Communicative Agents for Software Developmentcited by 893
- 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
- 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
- 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
- 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
- 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
- 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
- 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
- 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 580
- 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 550
- 2024Mixture-of-Agents Enhances Large Language Model Capabilitiescited by 420
- 2024InternLM2 Technical Reportcited by 391
- 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agentscited by 350
- 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMscited by 297
- 2023A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaborationcited by 281
- 2024Scaling Large Language Model-based Multi-Agent Collaborationcited by 235
- 2024RouterBench: A Benchmark for Multi-LLM Routing Systemcited by 222
- 2020ReClor: A Reading Comprehension Dataset Requiring Logical Reasoningcited by 211
- 2025AgentSociety: Large-Scale Simulation of LLM-Driven Generative Agents Advances Understanding of Human Behaviors and Societycited by 180
- 2023Large Language Model Routing with Benchmark Datasetscited by 166
- 2024Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systemscited by 129
- 2024G-Designer: Architecting Multi-agent Communication Topologies via Graph Neural Networkscited by 110
- 2024Improving Multi-Agent Debate with Sparse Communication Topologycited by 103
- 2025NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Modelcited by 94
- 2025Harnessing Multiple Large Language Models: A Survey on LLM Ensemblecited by 86
- 2025Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models?cited by 74
- 2023A Unified Theory of Diversity in Ensemble Learningcited by 71
- 2023Contrastive Decoding Improves Reasoning in Large Language Modelscited by 71
- 2024A Unified Approach to Routing and Cascading for LLMscited by 71
- 2024Ensemble Learning for Heterogeneous Large Language Models with Deep Parallel Collaborationcited by 66
- 2025BEST-Route: Adaptive LLM Routing with Test-Time Optimal Computecited by 63
- 2024Learning to Decode Collaboratively with Multiple Language Modelscited by 61
- 2024GenSim: A General Social Simulation Platform with Large Language Model based Agentscited by 48
50 most cited of 60 All 60 (JSON)
Built on by
Not cited yet.