# Beyond the All-in-One Agent: Benchmarking Role-Specialized Multi-Agent Collaboration in Enterprise Workflows

arXiv: https://arxiv.org/abs/2605.08761 · HTML: https://mlsanity.com/paper/2605.08761/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 12 of its 19 references that are in the map.
- First posted on arXiv 9 May 2026.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 12 of 12)

- [DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence](https://mlsanity.com/paper/2606.19348/) (2026)
- [OpenAI GPT-5 System Card](https://mlsanity.com/paper/2601.03267/) (2025)
- [The StarCraft Multi-Agent Challenge](https://mlsanity.com/paper/1902.04043/) (2019)
- [WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?](https://mlsanity.com/paper/2403.07718/) (2024)
- [TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks](https://mlsanity.com/paper/2412.14161/) (2024)
- [CRMArena: Understanding the Capacity of LLM Agents to Perform Professional CRM Tasks in Realistic Environments](https://mlsanity.com/paper/2411.02305/) (2024)
- [Silo-Bench: A Scalable Environment for Evaluating Distributed Coordination in Multi-Agent LLM Systems](https://mlsanity.com/paper/2603.01045/) (2026)
- [BrowserAgent: Building Web Agents with Human-Inspired Web Browsing Actions](https://mlsanity.com/paper/2510.10666/) (2025)
- [EnterpriseOps-Gym: Environments and Evaluations for Stateful Agentic Planning and Tool Use in Enterprise Settings](https://mlsanity.com/paper/2603.13594/) (2026)
- [COMMA: A Communicative Multimodal Multi-Agent Benchmark](https://mlsanity.com/paper/2410.07553/) (2024)
- [EntWorld: A Holistic Environment and Benchmark for Verifiable Enterprise GUI Agents](https://mlsanity.com/paper/2601.17722/) (2026)
- [EnterpriseLab: A Full-Stack Platform for developing and deploying agents in Enterprises](https://mlsanity.com/paper/2603.21630/) (2026)

## Built on by (most cited first, 2 of 2)

- [Beyond Corrected Memory: Execution Consistency in Multi-Agent Systems](https://mlsanity.com/paper/2610.08101/) (2026)
- [Where Knowledge and Authority Sit Changes What an Agent Benchmark Can Resolve](https://mlsanity.com/paper/2607.02975/) (2026)

## Abstract (arXiv)

Large language model (LLM) agents are increasingly expected to operate in enterprise environments, where work is distributed across specialized roles, permission-controlled systems, and cross-departmental procedures. However, existing enterprise benchmarks largely evaluate single agents with broad tool access, while existing multi-agent benchmarks rarely capture realistic enterprise constraints such as role specialization, access control, stateful business systems, and policy-based approvals. We introduce textscEntCollabBench, a benchmark for evaluating enterprise multi-agent collaboration. textscEntCollabBench simulates a permission-isolated organization with 11 role-specialized agents across six departments and contains two evaluation subsets: a Workflow subset, where agents collaboratively modify enterprise system states, and an Approval subset, where agents make policy-grounded decisions. Evaluation is based on execution traces, database state verification, and deterministic policy adjudication rather than natural-language response judging. Experiments with representative LLM agents show that current models still struggle with end-to-end enterprise collaboration, especially in delegation, context transfer, parameter grounding, workflow closure, and decision commitment. textscEntCollabBench provides a reproducible testbed for measuring and improving agent systems intended for realistic organizational environments.