# From Few to More: Large-scale Dynamic Multiagent Curriculum Learning

arXiv: https://arxiv.org/abs/1909.02790 · HTML: https://mlsanity.com/paper/1909.02790/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Sep 2019.

## Built on by (most cited first, 30 of 36)

- [UPDeT: Universal Multi-agent Reinforcement Learning via Policy Decoupling with Transformers](https://mlsanity.com/paper/2101.08001/) (2021)
- [Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Composition](https://mlsanity.com/paper/2105.08692/) (2021)
- [Variational Automatic Curriculum Learning for Sparse-Reward Cooperative Multi-Agent Problems](https://mlsanity.com/paper/2111.04613/) (2021)
- [Scalable Multi-Agent Model-Based Reinforcement Learning](https://mlsanity.com/paper/2205.15023/) (2022)
- [Learning Efficient Multi-Agent Cooperative Visual Exploration](https://mlsanity.com/paper/2110.05734/) (2021)
- [Contrastive Identity-Aware Learning for Multi-Agent Value Decomposition](https://mlsanity.com/paper/2211.12712/) (2022)
- [Asynchronous Multi-Agent Reinforcement Learning for Efficient Real-Time Multi-Robot Cooperative Exploration](https://mlsanity.com/paper/2301.03398/) (2023)
- [Multi-Agent Policy Transfer via Task Relationship Modeling](https://mlsanity.com/paper/2203.04482/) (2022)
- [Cooperative Multi-Agent Transfer Learning with Level-Adaptive Credit Assignment](https://mlsanity.com/paper/2106.00517/) (2021)
- [Transformers in Reinforcement Learning: A Survey](https://mlsanity.com/paper/2307.05979/) (2023)
- [Learning Cyber Defence Tactics from Scratch with Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2310.05939/) (2023)
- [Canaries and Whistles: Resilient Drone Communication Networks with (or without) Deep Reinforcement Learning](https://mlsanity.com/paper/2312.04940/) (2023)
- [Interaction Pattern Disentangling for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2207.03902/) (2022)
- [Learning Meta Representations for Agents in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2108.12988/) (2021)
- [Breaking the Curse of Dimensionality in Multiagent State Space: A Unified Agent Permutation Framework](https://mlsanity.com/paper/2203.05285/) (2022)
- [Heterogeneous Multi-Agent Reinforcement Learning for Zero-Shot Scalable Collaboration](https://mlsanity.com/paper/2404.03869/) (2024)
- [Learning to Transfer Role Assignment Across Team Sizes](https://mlsanity.com/paper/2204.12937/) (2022)
- [Towards Skilled Population Curriculum for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2302.03429/) (2023)
- [Learning Progress Driven Multi-Agent Curriculum](https://mlsanity.com/paper/2205.10016/) (2022)
- [DistSPECTRL: Distributing Specifications in Multi-Agent Reinforcement Learning Systems](https://mlsanity.com/paper/2206.13754/) (2022)
- [Learning Complex Teamwork Tasks Using a Given Sub-task Decomposition](https://mlsanity.com/paper/2302.04944/) (2023)
- [Hybrid Training for Enhanced Multi-task Generalization in Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2408.13567/) (2024)
- [Accelerate Multi-Agent Reinforcement Learning in Zero-Sum Games with Subgame Curriculum Learning](https://mlsanity.com/paper/2310.04796/) (2023)
- [SPECTra: Scalable Multi-Agent Reinforcement Learning with Permutation-Free Networks](https://mlsanity.com/paper/2503.11726/) (2025)
- [Meta-CPR: Generalize to Unseen Large Number of Agents with Communication Pattern Recognition Module](https://mlsanity.com/paper/2112.07222/) (2021)
- [CURO: Curriculum Learning for Relative Overgeneralization](https://mlsanity.com/paper/2212.02733/) (2022)
- [Hierarchical Task Network Planning for Facilitating Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2306.08359/) (2023)
- [A Dual Curriculum Learning Framework for Multi-UAV Pursuit-Evasion in Diverse Environments](https://mlsanity.com/paper/2312.12255/) (2023)
- [Self-Clustering Hierarchical Multi-Agent Reinforcement Learning with Extensible Cooperation Graph](https://mlsanity.com/paper/2403.18056/) (2024)
- [Beyond Shallow Behavior: Task-Efficient Value-Based Multi-Task Offline MARL via Skill Discovery](https://mlsanity.com/paper/2502.08985/) (2025)

## Abstract (arXiv)

A lot of efforts have been devoted to investigating how agents can learn effectively and achieve coordination in multiagent systems. However, it is still challenging in large-scale multiagent settings due to the complex dynamics between the environment and agents and the explosion of state-action space. In this paper, we design a novel Dynamic Multiagent Curriculum Learning (DyMA-CL) to solve large-scale problems by starting from learning on a multiagent scenario with a small size and progressively increasing the number of agents. We propose three transfer mechanisms across curricula to accelerate the learning process. Moreover, due to the fact that the state dimension varies across curricula,, and existing network structures cannot be applied in such a transfer setting since their network input sizes are fixed. Therefore, we design a novel network structure called Dynamic Agent-number Network (DyAN) to handle the dynamic size of the network input. Experimental results show that DyMA-CL using DyAN greatly improves the performance of large-scale multiagent learning compared with state-of-the-art deep reinforcement learning approaches. We also investigate the influence of three transfer mechanisms across curricula through extensive simulations.