# When Does Hierarchy Help? Benchmarking Agent Coordination in Event-Driven Industrial Scheduling

arXiv: https://arxiv.org/abs/2605.13172 · HTML: https://mlsanity.com/paper/2605.13172/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 32 references that are in the map.
- First posted on arXiv 13 May 2026.

## Builds on (most cited first, 25 of 25)

- [MultiAgentBench: Evaluating the Collaboration and Competition of LLM agents](https://mlsanity.com/paper/2503.01935/) (2025)
- [AgentScope: A Flexible yet Robust Multi-Agent Platform](https://mlsanity.com/paper/2402.14034/) (2024)
- [Multi-Agent Collaboration via Evolving Orchestration](https://mlsanity.com/paper/2505.19591/) (2025)
- [AgentNet: Decentralized Evolutionary Coordination for LLM-based Multi-Agent Systems](https://mlsanity.com/paper/2504.00587/) (2025)
- [Multi-Agent Coordination across Diverse Applications: A Survey](https://mlsanity.com/paper/2502.14743/) (2025)
- [HM-RAG: Hierarchical Multi-Agent Multimodal Retrieval Augmented Generation](https://mlsanity.com/paper/2504.12330/) (2025)
- [AgentsNet: Coordination and Collaborative Reasoning in Multi-Agent LLMs](https://mlsanity.com/paper/2507.08616/) (2025)
- [CoMAS: Co-Evolving Multi-Agent Systems via Interaction Rewards](https://mlsanity.com/paper/2510.08529/) (2025)
- [Agent AI with LangGraph: A Modular Framework for Enhancing Machine Translation Using Large Language Models](https://mlsanity.com/paper/2412.03801/) (2024)
- [BattleAgentBench: A Benchmark for Evaluating Cooperation and Competition Capabilities of Language Models in Multi-Agent Systems](https://mlsanity.com/paper/2408.15971/) (2024)
- [Deep Reinforcement Learning for Event-Driven Multi-Agent Decision Processes](https://mlsanity.com/paper/1709.06656/) (2017)
- [Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL](https://mlsanity.com/paper/2410.20254/) (2024)
- [AMAS: Adaptively Determining Communication Topology for LLM-based Multi-Agent System](https://mlsanity.com/paper/2510.01617/) (2025)
- [Multi-Agent Coordination via Multi-Level Communication](https://mlsanity.com/paper/2209.12713/) (2022)
- [A Taxonomy of Hierarchical Multi-Agent Systems: Design Patterns, Coordination Mechanisms, and Industrial Applications](https://mlsanity.com/paper/2508.12683/) (2025)
- [Hierarchical Decision Making Based on Structural Information Principles](https://mlsanity.com/paper/2404.09760/) (2024)
- [CP-AgentNet: Autonomous and Explainable Communication Protocol Design Using Generative Agents](https://mlsanity.com/paper/2503.17850/) (2025)
- [HYGMA: Hypergraph Coordination Networks with Dynamic Grouping for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2505.07207/) (2025)
- [Multi-Agent Guided Policy Optimization](https://mlsanity.com/paper/2507.18059/) (2025)
- [HiVA: Self-organized Hierarchical Variable Agent via Goal-driven Semantic-Topological Evolution](https://mlsanity.com/paper/2509.00189/) (2025)
- [MOSEAC: Streamlined Variable Time Step Reinforcement Learning](https://mlsanity.com/paper/2406.01521/) (2024)
- [Contextual Knowledge Sharing in Multi-Agent Reinforcement Learning with Decentralized Communication and Coordination](https://mlsanity.com/paper/2501.15695/) (2025)
- [A Production Scheduling Framework for Reinforcement Learning Under Real-World Constraints](https://mlsanity.com/paper/2506.13566/) (2025)
- [Hierarchical Decentralized Multi-Agent Coordination with Privacy-Preserving Knowledge Sharing: Extending AgentNet for Scalable Autonomous Systems](https://mlsanity.com/paper/2512.00614/) (2025)
- [Proposition of an implementation framework enabling benchmarking of Holonic Manufacturing Systems](https://mlsanity.com/paper/1901.05669/) (2019)

## Abstract (arXiv)

Recent advances in agent and multi-agent systems have shown strong performance on tool use, reasoning, and collaborative tasks. However, existing benchmarks mostly evaluate task completion in weakly coupled environments, and provide limited support for studying coordination in shared, dynamically evolving systems with hierarchy and coupled constraints. This leaves an important question underexplored: when do different coordination paradigms succeed or fail? We introduce Distributed Event-driven Scheduling Benchmark (DESBench), a benchmark for evaluating agent coordination in hierarchical event-driven scheduling. Built on a shared discrete-event driven environment in industrial scheduling, our benchmark captures multi-timescale decision making, partial observability, and dynamically coupled constraints. We define tasks and metrics that evaluate effectiveness, constraint alignment, coordination efficiency, and robustness, and focus on four representative coordination paradigms: centralized, hierarchical, heterarchical, and holonic. These paradigms correspond to distinct mechanisms of information flow, decision authority, and conflict resolution. Our controlled evaluations reveal clear coordination trade-offs: centralized coordination is robust and communication-efficient but scales poorly with difficulty; hierarchical coordination improves efficiency through decomposition but suffers from cross-level misalignment; heterarchical coordination is flexible but communication-heavy; and holonic coordination satisfies constraints well but loses global robustness. These findings demonstrate that coordination design fundamentally shapes agent system behavior in complex environments, revealing structural trade-offs that cannot be captured by outcome metrics alone and underscoring the imperative for more adaptive, principled, and dynamic coordination mechanisms in future MAS research.