# Improving Coordination in Small-Scale Multi-Agent Deep Reinforcement Learning through Memory-driven Communication

arXiv: https://arxiv.org/abs/1901.03887 · HTML: https://mlsanity.com/paper/1901.03887/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jan 2019.

## Built on by (most cited first, 18 of 18)

- [A Survey of Multi-Agent Deep Reinforcement Learning with Communication](https://mlsanity.com/paper/2203.08975/) (2022)
- [Discrete-Valued Neural Communication](https://mlsanity.com/paper/2107.02367/) (2021)
- [Collaborative Visual Navigation](https://mlsanity.com/paper/2107.01151/) (2021)
- [Distributed Reinforcement Learning for Robot Teams: A Review](https://mlsanity.com/paper/2204.03516/) (2022)
- [Emergent Language: A Survey and Taxonomy](https://mlsanity.com/paper/2409.02645/) (2024)
- [Adaptive Discrete Communication Bottlenecks with Dynamic Vector Quantization](https://mlsanity.com/paper/2202.01334/) (2022)
- [Multi-Agent Reinforcement Learning Based on Representational Communication for Large-Scale Traffic Signal Control](https://mlsanity.com/paper/2310.02435/) (2023)
- [Dynamic Collaborative Multi-Agent Reinforcement Learning Communication for Autonomous Drone Reforestation](https://mlsanity.com/paper/2211.15414/) (2022)
- [The Power of Communication in a Distributed Multi-Agent System](https://mlsanity.com/paper/2111.15611/) (2021)
- [Collaborative Auto-Curricula Multi-Agent Reinforcement Learning with Graph Neural Network Communication Layer for Open-ended Wildfire-Management Resource Distribution](https://mlsanity.com/paper/2204.11350/) (2022)
- [Dynamic Size Message Scheduling for Multi-Agent Communication under Limited Bandwidth](https://mlsanity.com/paper/2306.10134/) (2023)
- [Discrete Messages Improve Communication Efficiency among Isolated Intelligent Agents](https://mlsanity.com/paper/2312.15985/) (2023)
- [Mosaic: Runtime-Efficient Multi-Agent Embodied Planning](https://mlsanity.com/paper/2607.09603/) (2026)
- [CGIBNet: Bandwidth-constrained Communication with Graph Information Bottleneck in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2112.10374/) (2021)
- [CoMIX: A Multi-agent Reinforcement Learning Training Architecture for Efficient Decentralized Coordination and Independent Decision-Making](https://mlsanity.com/paper/2308.10721/) (2023)
- [Multi-agent Cooperative Games Using Belief Map Assisted Training](https://mlsanity.com/paper/2406.19477/) (2024)
- [Multi-Robot Pursuit in Parameterized Formation via Imitation Learning](https://mlsanity.com/paper/2410.23586/) (2024)
- [Q-MARL: A quantum-inspired algorithm using neural message passing for large-scale multi-agent reinforcement learning](https://mlsanity.com/paper/2503.07397/) (2025)

## Abstract (arXiv)

Deep reinforcement learning algorithms have recently been used to train multiple interacting agents in a centralised manner whilst keeping their execution decentralised. When the agents can only acquire partial observations and are faced with tasks requiring coordination and synchronisation skills, inter-agent communication plays an essential role. In this work, we propose a framework for multi-agent training using deep deterministic policy gradients that enables concurrent, end-to-end learning of an explicit communication protocol through a memory device. During training, the agents learn to perform read and write operations enabling them to infer a shared representation of the world. We empirically demonstrate that concurrent learning of the communication device and individual policies can improve inter-agent coordination and performance in small-scale systems. Our experimental results show that the proposed method achieves superior performance in scenarios with up to six agents. We illustrate how different communication patterns can emerge on six different tasks of increasing complexity. Furthermore, we study the effects of corrupting the communication channel, provide a visualisation of the time-varying memory content as the underlying task is being solved and validate the building blocks of the proposed memory device through ablation studies.