ClawArena-Team: Benchmarking Subagent Orchestration and Dynamic Workflows in Language-Model Agents
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
Production large language-model (LLM) agents are increasingly deployed not as lone problem-solvers but as managers: a main model creates specialized subagents, delegates work, and orchestrates their parallel, asynchronous returns through dynamic workflows. Whether one model can actually run such a team is largely unmeasured: existing benchmarks score a policy's own task-solving or a fixed multi-agent system's emergent behavior, but none isolate the management ability of the single LLM acting as leader. We introduce ClawArena-Team, a benchmark of 41 multi-turn, multimodal, multi-directory scenarios spanning 258 evaluation rounds and 72 staged updates that measures this management ability. The main agent is deliberately constrained: it natively perceives only text and directly accesses only part of the workspace. It commands a fixed, locally served subagent pool, so score differences reflect management skill, not raw capability. All scoring is execution-based with no LLM judge: an overall score -- the Subagent-Management Score (SMS) -- multiplies task correctness by a least-privilege and modality-routing factor. Across twelve proprietary, community-hosted, and self-hosted models, experiments show that the management bottleneck is privilege granting rather than perception (no model exceeds 50% workspace-permission precision); that cost and management quality are decoupled (API cost spans over 100 times while the overall score spans under 4 times, with the cheapest open models on the Pareto frontier); and that most leaderboard scores cluster within a 9.9-point band while orchestration behaviors diverge by more than an order of magnitude. Code is available at https://github.com/aiming-lab/ClawArena.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 23 of its 29 references that are in the map. First posted on arXiv 30 Jun 2026.
Builds on
23 of 29 references.
Closest ancestors · 11
Same direction, newest first
- 2026ClawArena: Benchmarking AI Agents in Evolving Information Environments5 Apr
- 2025AgentOrchestra: Orchestrating Multi-Agent Intelligence with the Tool-Environment-Agent(TEA) Protocol14 Jun
- 2025DynTaskMAS: A Dynamic Task Graph-driven Framework for Asynchronous and Parallel LLM-based Multi-Agent Systems10 Mar
- 2025MultiAgentBench: Evaluating the Collaboration and Competition of LLM agents3 Mar
- 2024Collaborative Gym: A Framework for Enabling and Evaluating Human-Agent Collaboration20 Dec
- 2024TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks18 Dec
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains17 Jun
- 2024MLLM-Tool: A Multimodal Large Language Model For Tool Agent Learning19 Jan
- 2023AgentBench: Evaluating LLMs as Agents7 Aug
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework1 Aug
- 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society31 Mar
Other references · 12
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
- 2023Large Language Modelscited by 780
- 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 682
- 2025Why Do Multi-Agent LLM Systems Fail?cited by 604
- 2024RouterBench: A Benchmark for Multi-LLM Routing Systemcited by 238
- 2025Progent: Securing AI Agents with Privilege Controlcited by 141
- 2024ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agentscited by 133
- 2026Taming Various Privilege Escalation in LLM-Based Agent Systems: A Mandatory Access Control Frameworkcited by 54
- 2025Orchestrating Human-AI Teams: The Manager Agent as a Unifying Research Challengecited by 8
- 2025MEXA: Towards General Multimodal Reasoning with Dynamic Multi-Expert Aggregationcited by 6
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments
- 2026BabelArena: A Large-Scale Multilingual Benchmark for LLM Agents
- 2026UniACE: A Unified Framework for Evaluating LLM Agentic Capabilities
- 2026Improving the Efficiency of Language Agent Teams with Adaptive Task Graphs
- 2026Beyond the Leaderboard: A Synthesis of Tool-Use, Planning, and Reasoning Failures in Large Language Model Agents
- 2026DynamicMCPBench: A Trace-Grounded, Effect-Scored Benchmark for LLM Agents over Live MCP Servers
- 2026How Fast Do Agents Rot? An Empirical Study of Long-Horizon Degradation in LLM Agents for Production Decision-Making
- 2026Agent Gym: A Framework for Continuous Evaluation and Evolution of LLM Agents Through Human-in-the-Loop Feedback