# A Decentralized Communication Framework based on Dual-Level Recurrence for Multi-Agent Reinforcement Learning

arXiv: https://arxiv.org/abs/2202.10612 · HTML: https://mlsanity.com/paper/2202.10612/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 39 references that are in the map.
- First posted on arXiv 22 Feb 2022.

## Builds on (most cited first, 16 of 16)

- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms](https://mlsanity.com/paper/1911.10635/) (2019)
- [Learning Multiagent Communication with Backpropagation](https://mlsanity.com/paper/1605.07736/) (2016)
- [Actor-Attention-Critic for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1810.02912/) (2018)
- [TarMAC: Targeted Multi-Agent Communication](https://mlsanity.com/paper/1810.11187/) (2018)
- [Learning Attentional Communication for Multi-Agent Cooperation](https://mlsanity.com/paper/1805.07733/) (2018)
- [When2com: Multi-Agent Perception via Communication Graph Grouping](https://mlsanity.com/paper/2006.00176/) (2020)
- [CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenario](https://mlsanity.com/paper/1905.05217/) (2019)
- [Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Games](https://mlsanity.com/paper/1703.10069/) (2017)
- [Large-Scale Traffic Signal Control Using a Novel Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1908.03761/) (2019)
- [Managing engineering systems with large state and action spaces through deep reinforcement learning](https://mlsanity.com/paper/1811.02052/) (2018)
- [Learning to Communicate to Solve Riddles with Deep Distributed Recurrent Q-Networks](https://mlsanity.com/paper/1602.02672/) (2016)
- [Information State Embedding in Partially Observable Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2004.01098/) (2020)
- [Multi-agent Reinforcement Learning for Resource Allocation in IoT networks with Edge Computing](https://mlsanity.com/paper/2004.02315/) (2020)
- [Distributed Cooperative Spectrum Sharing in UAV Networks Using Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1811.05053/) (2018)

## Abstract (arXiv)

We propose a model enabling decentralized multiple agents to share their perception of environment in a fair and adaptive way. In our model, both the current message and historical observation are taken into account, and they are handled in the same recurrent model but in different forms. We present a dual-level recurrent communication framework for multi-agent systems, in which the first recurrence occurs in the communication sequence and is used to transmit communication data among agents, while the second recurrence is based on the time sequence and combines the historical observations for each agent. The developed communication flow separates communication messages from memories but allows agents to share their historical observations by the dual-level recurrence. This design makes agents adapt to changeable communication objects, while the communication results are fair to these agents. We provide a sufficient discussion about our method in both partially observable and fully observable environments. The results of several experiments suggest our method outperforms the existing decentralized communication frameworks and the corresponding centralized training method.