# Learning Structured Communication for Multi-agent Reinforcement Learning

arXiv: https://arxiv.org/abs/2002.04235 · HTML: https://mlsanity.com/paper/2002.04235/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Feb 2020.

## Built on by (most cited first, 21 of 21)

- [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023)
- [A Survey of Multi-Agent Deep Reinforcement Learning with Communication](https://mlsanity.com/paper/2203.08975/) (2022)
- [Internet of Agents: Fundamentals, Applications, and Challenges](https://mlsanity.com/paper/2505.07176/) (2025)
- [AdaSociety: An Adaptive Environment with Social Structures for Multi-Agent Decision-Making](https://mlsanity.com/paper/2411.03865/) (2024)
- [HAMMER: Multi-Level Coordination of Reinforcement Learning Agents via Learned Messaging](https://mlsanity.com/paper/2102.00824/) (2021)
- [Hierarchical Consensus-Based Multi-Agent Reinforcement Learning for Multi-Robot Cooperation Tasks](https://mlsanity.com/paper/2407.08164/) (2024)
- [Generative Multi-Agent Collaboration in Embodied AI: A Systematic Review](https://mlsanity.com/paper/2502.11518/) (2025)
- [LERO: LLM-driven Evolutionary framework with Hybrid Rewards and Enhanced Observation for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2503.21807/) (2025)
- [TodyComm: Task-Oriented Dynamic Communication for Multi-Round LLM-based Multi-Agent System](https://mlsanity.com/paper/2602.03688/) (2026)
- [Learning Roles with Emergent Social Value Orientations](https://mlsanity.com/paper/2301.13812/) (2023)
- [Robust and Efficient Communication in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2511.11393/) (2025)
- [Structured Diversification Emergence via Reinforced Organization Control and Hierarchical Consensus Learning](https://mlsanity.com/paper/2102.04775/) (2021)
- [AET-SGD: Asynchronous Event-triggered Stochastic Gradient Descent](https://mlsanity.com/paper/2112.13935/) (2021)
- [Toward Adaptable Multi-Agent Reinforcement Learning: An Assumption-Aware Review](https://mlsanity.com/paper/2507.10142/) (2025)
- [HiComm: Hierarchical Communication for Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2606.29126/) (2026)
- [A Cooperative Reinforcement Learning Environment for Detecting and Penalizing Betrayal](https://mlsanity.com/paper/2210.12841/) (2022)
- [Collective Large-scale Wind Farm Multivariate Power Output Control Based on Hierarchical Communication Multi-Agent Proximal Policy Optimization](https://mlsanity.com/paper/2305.10161/) (2023)
- [Multi-agent Cooperative Games Using Belief Map Assisted Training](https://mlsanity.com/paper/2406.19477/) (2024)
- [Interference-Aware K-Step Reachable Communication in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2603.15054/) (2026)
- [A Survey of Multi-Agent Deep Reinforcement Learning with Graph Neural Network-Based Communication](https://mlsanity.com/paper/2604.25972/) (2026)
- [GCT-MARL: Graph-Based Contrastive Transfer for Sample-Efficient Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2606.25073/) (2026)

## Abstract (arXiv)

This work explores the large-scale multi-agent communication mechanism under a multi-agent reinforcement learning (MARL) setting. We summarize the general categories of topology for communication structures in MARL literature, which are often manually specified. Then we propose a novel framework termed as Learning Structured Communication (LSC) by using a more flexible and efficient communication topology. Our framework allows for adaptive agent grouping to form different hierarchical formations over episodes, which is generated by an auxiliary task combined with a hierarchical routing protocol. Given each formed topology, a hierarchical graph neural network is learned to enable effective message information generation and propagation among inter- and intra-group communications. In contrast to existing communication mechanisms, our method has an explicit while learnable design for hierarchical communication. Experiments on challenging tasks show the proposed LSC enjoys high communication efficiency, scalability, and global cooperation capability.