mlsanityData to 7 Oct 2026

Which Models Work Well Together? Measuring Heterogeneity for LLM Team Selection

arXiv 2609.38274 ↗First posted 29 Sep 2026cs.CL cs.LG cs.MAcited by 0

Not on the map. Why?

Abstract (arXiv)

The performance ceiling of an LLM team is constrained not only by individual model capabilities, but also by inter-member error resonance and predictive differences. Although heterogeneous teaming is often observed to be effective in practice, existing approaches lack complementarity metrics that are computable, interpretable, and optimizable, leaving team composition to rely on heuristics. We propose a heterogeneity-driven team selection framework that performs offline profiling to characterize individual capability along with two complementary signals: one captures decorrelation in error patterns to reduce co-failures, while the other measures divergence in predictive behavior to capture strategy diversity. We formulate team selection as a standardized quality--complementarity combinatorial objective and apply an efficient greedy search to select a small team from a candidate pool. Experiments across multiple benchmarks demonstrate that our framework consistently outperforms quality-only baselines under controlled candidate pools and team sizes, establishing reusable selection principles for multi-LLM systems.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 60 of its 75 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

60 of 75 references.

References · 60

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  5. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  6. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  7. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  8. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  9. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  10. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  11. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  12. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
38 more
  1. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  2. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  3. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  4. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  5. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  6. 2023ChatDev: Communicative Agents for Software Developmentcited by 893
  7. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
  8. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
  9. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  10. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  11. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
  12. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  13. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
  14. 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 580
  15. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 550
  16. 2024Mixture-of-Agents Enhances Large Language Model Capabilitiescited by 420
  17. 2024InternLM2 Technical Reportcited by 391
  18. 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agentscited by 350
  19. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMscited by 297
  20. 2023A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaborationcited by 281
  21. 2024Scaling Large Language Model-based Multi-Agent Collaborationcited by 235
  22. 2024RouterBench: A Benchmark for Multi-LLM Routing Systemcited by 222
  23. 2020ReClor: A Reading Comprehension Dataset Requiring Logical Reasoningcited by 211
  24. 2025AgentSociety: Large-Scale Simulation of LLM-Driven Generative Agents Advances Understanding of Human Behaviors and Societycited by 180
  25. 2023Large Language Model Routing with Benchmark Datasetscited by 166
  26. 2024Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systemscited by 129
  27. 2024G-Designer: Architecting Multi-agent Communication Topologies via Graph Neural Networkscited by 110
  28. 2024Improving Multi-Agent Debate with Sparse Communication Topologycited by 103
  29. 2025NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Modelcited by 94
  30. 2025Harnessing Multiple Large Language Models: A Survey on LLM Ensemblecited by 86
  31. 2025Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models?cited by 74
  32. 2023A Unified Theory of Diversity in Ensemble Learningcited by 71
  33. 2023Contrastive Decoding Improves Reasoning in Large Language Modelscited by 71
  34. 2024A Unified Approach to Routing and Cascading for LLMscited by 71
  35. 2024Ensemble Learning for Heterogeneous Large Language Models with Deep Parallel Collaborationcited by 66
  36. 2025BEST-Route: Adaptive LLM Routing with Test-Time Optimal Computecited by 63
  37. 2024Learning to Decode Collaboratively with Multiple Language Modelscited by 61
  38. 2024GenSim: A General Social Simulation Platform with Large Language Model based Agentscited by 48

50 most cited of 60 All 60 (JSON)

Built on by

Not cited yet.