# CTC: The Composite Task Challenge for Cooperative Multi-Agent Reinforcement Learning

arXiv: https://arxiv.org/abs/2502.00345 · HTML: https://mlsanity.com/paper/2502.00345/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 29 references that are in the map.
- First posted on arXiv 1 Feb 2025.

## Builds on (most cited first, 20 of 20)

- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [The StarCraft Multi-Agent Challenge](https://mlsanity.com/paper/1902.04043/) (2019)
- [Emergence of Grounded Compositional Language in Multi-Agent Populations](https://mlsanity.com/paper/1703.04908/) (2017)
- [Google Research Football: A Novel Reinforcement Learning Environment](https://mlsanity.com/paper/1907.11180/) (2019)
- [Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks](https://mlsanity.com/paper/2006.07869/) (2020)
- [MAVEN: Multi-Agent Variational Exploration](https://mlsanity.com/paper/1910.07483/) (2019)
- [ROMA: Multi-Agent Reinforcement Learning with Emergent Roles](https://mlsanity.com/paper/2003.08039/) (2020)
- [RODE: Learning Roles to Decompose Multi-Agent Tasks](https://mlsanity.com/paper/2010.01523/) (2020)
- [SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2212.07489/) (2022)
- [Shared Experience Actor-Critic for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2006.07169/) (2020)
- [Celebrating Diversity in Shared Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2106.02195/) (2021)
- [Scaling Multi-Agent Reinforcement Learning with Selective Parameter Sharing](https://mlsanity.com/paper/2102.07475/) (2021)
- [Rethinking the Implementation Tricks and Monotonicity Constraint in Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2102.03479/) (2021)
- [Hierarchical Cooperative Multi-Agent Reinforcement Learning with Skill Discovery](https://mlsanity.com/paper/1912.03558/) (2019)
- [The Emergence of Individuality](https://mlsanity.com/paper/2006.05842/) (2020)
- [Group-Aware Coordination Graph for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2404.10976/) (2024)
- [LDSA: Learning Dynamic Subtask Assignment in Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2205.02561/) (2022)
- [QTypeMix: Enhancing Multi-Agent Cooperative Strategies through Heterogeneous and Homogeneous Value Decomposition](https://mlsanity.com/paper/2408.07098/) (2024)

## Abstract (arXiv)

The critical role of division of labor (DOL) in enhancing cooperation is well-recognized in real-world applications. Consequently, many cooperative multi-agent reinforcement learning (MARL) methods have incorporated DOL mechanisms to improve cooperation among agents. However, the lack of benchmark tasks specifically designed to evaluate and promote DOL and cooperation has limited the effective development and deployment of such mechanisms in cooperative MARL. This gap between current cooperative MARL methods and practical applications underscores the need for evaluation tasks that explicitly require DOL and cooperation. To address this gap, we propose the Composite Tasks Challenge (CTC), a suite of tasks explicitly designed to require both DOL and cooperation for successful task completion. The CTC tasks are constructed based on two core design principles: 1) DOL is a necessary condition for task success; 2) Failure in any atomic subtask results in failure of the overall task. The first principle emphasizes the necessity of DOL, while the second enforces the importance of cooperation, making both components essential for success in CTC tasks. We evaluate nine representative cooperative MARL methods on the proposed CTC tasks. Experimental results show that all methods consistently achieve zero test winning rates across all CTC tasks, highlighting the challenge of CTC tasks and the limitations of current methods. To facilitate future research, we also introduce a guiding solution that achieves non-zero test winning rates on all tasks, thereby demonstrating the solvability of the CTC tasks. However, the performance of this guiding solution remains suboptimal, further underscoring the value of CTC tasks as a challenging and meaningful testbed for advancing cooperative MARL research.