mlsanityData to 14 Oct 2026

Beyond the Strongest LLM: Multi-Turn Multi-Agent Orchestration vs. Single LLMs on Benchmarks

arXiv 2509.23537 ↗First posted 28 Sep 2025cs.AIcited by 9

On the map: LLM agents and tool use → Social simulation and multi-agent debate

Abstract (arXiv)

We study multi-turn multi-agent orchestration, where multiple large language model (LLM) agents interact over multiple turns by iteratively proposing answers or casting votes until reaching consensus. Using four LLMs (Gemini 2.5 Pro, GPT-5, Grok 4, and Claude Sonnet 4) on GPQA-Diamond, IFEval, and MuSR, we conduct two experiments: (i) benchmarking orchestration against single-LLM baselines; and (ii) ablations on GPQA-Diamond that vary whether agents see who authored answers and whether they can observe ongoing votes. Orchestration matches or exceeds the strongest single model and consistently outperforms the others. Analysis of best-achievable orchestration performance shows potential for further gains. The ablations show that revealing authorship increases self-voting and ties, and that showing ongoing votes amplifies herding, which speeds convergence but can sometimes yield premature consensus.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 7 of its 10 references that are in the map. First posted on arXiv 28 Sep 2025.

Builds on

7 of 10 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025From Debate to Equilibrium: Belief-Driven Multi-Agent LLM Reasoning via Bayesian Nash Equilibrium9 Jun
  2. 2025Voting or Consensus? Decision-Making in Multi-Agent Debate26 Feb
  3. 2023Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View3 Oct

Other references · 4

  1. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  2. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 431
  3. 2025MAPoRL: Multi-Agent Post-Co-Training for Collaborative Large Language Models with Reinforcement Learningcited by 86
  4. 2025A Strategic Coordination Framework of Small LLMs Matches Large LLMs in Data Synthesiscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 8.

9 papers.

  1. 2026Internet of Agentic AI: Incentive-Compatible Distributed Teaming and Workflowcited by 8
  2. 2026Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Modelcited by 3
  3. 2026Heterogeneous Scientific Foundation Model Collaborationcited by 1
  4. 2026OrchestraBench: Evaluating Multi-Agent Orchestration Failure Modes, Recovery, and Decomposition Qualitycited by 1
  5. 2025The ORCA Benchmark: Evaluating Real-World Calculation Accuracy in Large Language Modelscited by 0
  6. 2026Agents in the Wild: Where Research Meets Deploymentcited by 0
  7. 2026A MARL Centered Reference Architecture for Large Language Model Augmentation in Smart Manufacturingcited by 0
  8. 2026Mint-Agent: Introducing Finance-Native Agentic Foundation Modelscited by 0
  9. 2026Right-Sizing LLM-Agent Decomposition in VAT Determination: A Pilot Controlled Sweepcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Beyond Majority Voting: LLM Aggregation by Leveraging Higher-Order Information
  2. 2025Can LLM Agents Really Debate? A Controlled Study of Multi-Agent Debate in Logical Reasoning
  3. 2026Representational Collapse in Multi-Agent LLM Committees: Measurement and Diversity-Aware Consensus
  4. 2026The Consensus Trap: Rescuing Multi-Agent LLMs from Adversarial Majorities via Token-Level Collaboration
  5. 2026Sequential Consensus for Multi-Agent LLM Debates: A Wald-SPRT compute governor with calibration-based failure detection
  6. 2023Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMs
  7. 2025Debate Only When Necessary: Adaptive Multiagent Collaboration for Efficient LLM Reasoning
  8. 2026Can Small Agents Collaborate to Beat a Single Large Language Model?