# Learning Transferable Cooperative Behavior in Multi-Agent Teams

arXiv: https://arxiv.org/abs/1906.01202 · HTML: https://mlsanity.com/paper/1906.01202/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jun 2019.

## Built on by (most cited first, 30 of 44)

- [A Survey of Multi-Agent Deep Reinforcement Learning with Communication](https://mlsanity.com/paper/2203.08975/) (2022)
- [Towards a Standardised Performance Evaluation Protocol for Cooperative MARL](https://mlsanity.com/paper/2209.10485/) (2022)
- [Scalable Multi-Agent Reinforcement Learning through Intelligent Information Aggregation](https://mlsanity.com/paper/2211.02127/) (2022)
- [Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Composition](https://mlsanity.com/paper/2105.08692/) (2021)
- [More Centralized Training, Still Decentralized Execution: Multi-Agent Conditional Policy Factorization](https://mlsanity.com/paper/2209.12681/) (2022)
- [Distributed Reinforcement Learning for Robot Teams: A Review](https://mlsanity.com/paper/2204.03516/) (2022)
- [Divergence-Regularized Multi-Agent Actor-Critic](https://mlsanity.com/paper/2110.00304/) (2021)
- [Generative Emergent Communication: Large Language Model is a Collective World Model](https://mlsanity.com/paper/2501.00226/) (2024)
- [Multi-Agent Policy Transfer via Task Relationship Modeling](https://mlsanity.com/paper/2203.04482/) (2022)
- [Cooperative Multi-Agent Transfer Learning with Level-Adaptive Credit Assignment](https://mlsanity.com/paper/2106.00517/) (2021)
- [Boosting Sample Efficiency and Generalization in Multi-agent Reinforcement Learning via Equivariance](https://mlsanity.com/paper/2410.02581/) (2024)
- [TransfQMix: Transformers for Leveraging the Graph Structure of Multi-Agent Reinforcement Learning Problems](https://mlsanity.com/paper/2301.05334/) (2023)
- [SRMT: Shared Memory for Multi-agent Lifelong Pathfinding](https://mlsanity.com/paper/2501.13200/) (2025)
- [Learning Generalizable Skills from Offline Multi-Task Data for Multi-Agent Cooperation](https://mlsanity.com/paper/2503.21200/) (2025)
- [Concentration Network for Reinforcement Learning of Large-Scale Multi-Agent Systems](https://mlsanity.com/paper/2203.06416/) (2022)
- [Multi-Agent Reinforcement Learning Based on Representational Communication for Large-Scale Traffic Signal Control](https://mlsanity.com/paper/2310.02435/) (2023)
- [Learning Meta Representations for Agents in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2108.12988/) (2021)
- [Interpretable Learned Emergent Communication for Human-Agent Teams](https://mlsanity.com/paper/2201.07452/) (2022)
- [Heterogeneous Multi-Agent Reinforcement Learning for Zero-Shot Scalable Collaboration](https://mlsanity.com/paper/2404.03869/) (2024)
- [Solving Multi-Agent Safe Optimal Control with Distributed Epigraph Form MARL](https://mlsanity.com/paper/2504.15425/) (2025)
- [MA2QL: A Minimalist Approach to Fully Decentralized Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2209.08244/) (2022)
- [Decentralized Policy Optimization](https://mlsanity.com/paper/2211.03032/) (2022)
- [Graph learning in robotics: a survey](https://mlsanity.com/paper/2310.04294/) (2023)
- [Cooperation and Fairness in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2410.14916/) (2024)
- [Towards True Lossless Sparse Communication in Multi-Agent Systems](https://mlsanity.com/paper/2212.00115/) (2022)
- [Measuring Policy Distance for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2401.11257/) (2024)
- [On the Role of Emergent Communication for Social Learning in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2302.14276/) (2023)
- [Universally Expressive Communication in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2206.06758/) (2022)
- [Generalization of Heterogeneous Multi-Robot Policies via Awareness and Communication of Capabilities](https://mlsanity.com/paper/2401.13127/) (2024)
- [Hierarchical Message-Passing Policies for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2507.23604/) (2025)

## Abstract (arXiv)

While multi-agent interactions can be naturally modeled as a graph, the environment has traditionally been considered as a black box. We propose to create a shared agent-entity graph, where agents and environmental entities form vertices, and edges exist between the vertices which can communicate with each other. Agents learn to cooperate by exchanging messages along the edges of this graph. Our proposed multi-agent reinforcement learning framework is invariant to the number of agents or entities present in the system as well as permutation invariance, both of which are desirable properties for any multi-agent system representation. We present state-of-the-art results on coverage, formation and line control tasks for multi-agent teams in a fully decentralized framework and further show that the learned policies quickly transfer to scenarios with different team sizes along with strong zero-shot generalization performance. This is an important step towards developing multi-agent teams which can be realistically deployed in the real world without assuming complete prior knowledge or instantaneous communication at unbounded distances.