PerspectiveGap: A Benchmark for Multi-Agent Orchestration Prompting
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
Real-world LLM applications are moving beyond single-agent workflows toward orchestrated multi-agent systems, yet current models still struggle to determine what each sub-agent needs to know. To measure this, we introduce PerspectiveGap, a benchmark for evaluating LLMs' ability to compose orchestration prompts for multi-agent systems. PerspectiveGap contains 110 scenarios, each evaluated through two distractor-mixed task formats: role-fragment assignment and free-form prompt writing. These scenarios are organized into 10 topologies, which are distilled from the authors' real-world engineering practice and framed by the Prompt Economy principle: building loop-centered orchestrations that maximize utility with minimal role and engineering overhead. In experiments with 33 commercial models from 10 companies, GPT-5.5 substantially outperforms all competitors, whereas Opus 4.8 shows a notable weakness in orchestration prompting despite its strong coding performance. Nevertheless, PerspectiveGap remains challenging: the evaluated models achieve an average combined pass rate of only 17.2% (GPT-5.5 62.0%) and an average overall leakage rate of 217.9% (a per-scenario information leak-event count, not a proportion; GPT-5.5 49.1%). These findings suggest that multi-agent orchestration prompting is a distinct and under-evaluated capability, and PerspectiveGap provides a foundation for measuring and improving it systematically.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 27 of its 44 references that are in the map. First posted on arXiv 7 Jun 2026.
Builds on
27 of 44 references.
Closest ancestors · 12
Same direction, newest first
- 2026DecisionBench: A Benchmark for Emergent Delegation in Long-Horizon Agentic Workflows18 May
- 2026Agent-BRACE: Decoupling Beliefs from Actions in Long-Horizon Tasks via Verbalized State Uncertainty12 May
- 2026Silo-Bench: A Scalable Environment for Evaluating Distributed Coordination in Multi-Agent LLM Systems1 Mar
- 2026Agentic Design Patterns: A System-Theoretic Framework27 Jan
- 2025OSC: Cognitive Orchestration through Dynamic Knowledge Alignment in Multi-Agent LLM Collaboration5 Sep
- 2025AGENTIF: Benchmarking Instruction Following of Large Language Models in Agentic Scenarios22 May
- 2025Systematic Failures in Collective Reasoning under Distributed Information in Multi-Agent LLMs15 May
- 2025OptimAI: Optimization from Natural Language Using LLM-Powered AI Agents23 Apr
- 2025SagaLLM: Context Management, Validation, and Transaction Guarantees for Multi-Agent LLM Planning15 Mar
- 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMs10 Jan
- 2024Agent Design Pattern Catalogue: A Collection of Architectural Patterns for Foundation Model based Agents16 May
- 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society31 Mar
Other references · 15
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
- 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,071
- 2025Why Do Multi-Agent LLM Systems Fail?cited by 604
- 2023Understanding Social Reasoning in Language Models with Language Modelscited by 172
- 2023FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactionscited by 171
- 2023Think Twice: Perspective-Taking Improves Large Language Models' Theory-of-Mind Capabilitiescited by 89
- 2024SimpleToM: Exposing the Gap between Explicit ToM Inference and Implicit ToM Application in LLMscited by 57
- 2025SWE-Bench++: A Framework for the Scalable Generation of Software Engineering Benchmarks from Open-Source Repositoriescited by 28
- 2025SPIN-Bench: How Well Do LLMs Plan Strategically and Reason Socially?cited by 24
- 2025Multi-Agent Tool-Integrated Policy Optimizationcited by 16
- 2026TeamBench: Evaluating Agent Coordination under Enforced Role Separationcited by 10
3 more
- 2026AutoNumerics: An Autonomous, PDE-Agnostic Multi-Agent Pipeline for Scientific Computingcited by 8
- 2026ReSearch: A Multi-Stage Machine Learning Framework for Earth Science Data Discoverycited by 2
- 2026Selective Deficits in LLM Mental Self-Modeling in a Behavior-Based Test of Theory of Mindcited by 2
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026Agon: An Autonomous Large-Scale Omnidisciplinary Research System Built on Prompt Economycited by 4
- 2026Understanding Sparse Attention Selectivity in Long-Context Foundation Models via Counterfactual Evaluationcited by 1
- 2026When Stories Evolve: Benchmarking LLM Storytelling Across Agent Architectures in Open-Ended World Simulationscited by 1
- 2026AgonAlpha: Autonomous Alpha Discovery via Prompt Economy and Scalable Agentic Searchcited by 0
- 2026AutoSR: Automatic Symbolic Regression by Searching Research Statescited by 0
- 2026Scoring the Wrong Question: Readout Failures in Constrained-Option Evaluationcited by 0
- 2026Scoring the Wrong Question: Readout Failures in Constrained-Option Evaluation27 Sep 2026
- 2026AutoSR: Automatic Symbolic Regression by Searching Research States17 Aug 2026
- 2026When Stories Evolve: Benchmarking LLM Storytelling Across Agent Architectures in Open-Ended World Simulations16 Aug 2026
- 2026AgonAlpha: Autonomous Alpha Discovery via Prompt Economy and Scalable Agentic Search4 Aug 2026
- 2026Understanding Sparse Attention Selectivity in Long-Context Foundation Models via Counterfactual Evaluation3 Aug 2026
- 2026Agon: An Autonomous Large-Scale Omnidisciplinary Research System Built on Prompt Economy23 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2026Utility-Guided Agent Orchestration for Efficient LLM Tool Use
- 2026Compiling Agentic Workflows into LLM Weights: Near-Frontier Quality at Two Orders of Magnitude Less Cost
- 2026Empirical Comparison of Agent Communication Protocols for Task Orchestration
- 2026Orchestra-o1: Omnimodal Agent Orchestration
- 2026ClawArena-Team: Benchmarking Subagent Orchestration and Dynamic Workflows in Language-Model Agents
- 2025Multi-Agent LLM Orchestration Achieves Deterministic, High-Quality Decision Support for Incident Response
- 2026How Generation Architecture Shapes Code Complexity in Multi-Agent LLM Systems: A Paired Study on HumanEval
- 2026Team of Thoughts: Efficient Test-time Scaling of Agentic Systems through Orchestrated Tool Calling