# Communication-Enhanced Tutoring for Efficient Decentralized Multi-Agent Reinforcement Learning

arXiv: https://arxiv.org/abs/2508.13661 · HTML: https://mlsanity.com/paper/2508.13661/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 44 references that are in the map.
- First posted on arXiv 19 Aug 2025.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Linformer: Self-Attention with Linear Complexity](https://mlsanity.com/paper/2006.04768/) (2020)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [The StarCraft Multi-Agent Challenge](https://mlsanity.com/paper/1902.04043/) (2019)
- [Learning by Cheating](https://mlsanity.com/paper/1912.12294/) (2019)
- [QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1905.05408/) (2019)
- [Learning Multiagent Communication with Backpropagation](https://mlsanity.com/paper/1605.07736/) (2016)
- [QPLEX: Duplex Dueling Multi-Agent Q-Learning](https://mlsanity.com/paper/2008.01062/) (2020)
- [Reinforcement learning](https://mlsanity.com/paper/2405.10369/) (2024)
- [Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks](https://mlsanity.com/paper/2006.07869/) (2020)
- [Weighted QMIX: Expanding Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2006.10800/) (2020)
- [Multi-Agent Reinforcement Learning is a Sequence Modeling Problem](https://mlsanity.com/paper/2205.14953/) (2022)
- [Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications](https://mlsanity.com/paper/1812.11794/) (2018)
- [Celebrating Diversity in Shared Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2106.02195/) (2021)
- [Learning Nearly Decomposable Value Functions Via Communication Minimization](https://mlsanity.com/paper/1910.05366/) (2019)
- [Multi-Agent Game Abstraction via Graph Attention Neural Network](https://mlsanity.com/paper/1911.10715/) (2019)
- [Qatten: A General Framework for Cooperative Multiagent Reinforcement Learning](https://mlsanity.com/paper/2002.03939/) (2020)
- [Single and Multi-Agent Deep Reinforcement Learning for AI-Enabled Wireless Networks: A Tutorial](https://mlsanity.com/paper/2011.03615/) (2020)
- [Learning Multi-Agent Communication from Graph Modeling Perspective](https://mlsanity.com/paper/2405.08550/) (2024)
- [Bridging the Imitation Gap by Adaptive Insubordination](https://mlsanity.com/paper/2007.12173/) (2020)
- [Applications of Multi-Agent Reinforcement Learning in Future Internet: A Comprehensive Survey](https://mlsanity.com/paper/2110.13484/) (2021)
- [SHAQ: Incorporating Shapley Value Theory into Multi-Agent Q-Learning](https://mlsanity.com/paper/2105.15013/) (2021)
- [Robust Asymmetric Learning in POMDPs](https://mlsanity.com/paper/2012.15566/) (2020)
- [Decentralized Transformers with Centralized Aggregation are Sample-Efficient Multi-Agent World Models](https://mlsanity.com/paper/2406.15836/) (2024)
- [Scalable Communication for Multi-Agent Reinforcement Learning via Transformer-Based Email Mechanism](https://mlsanity.com/paper/2301.01919/) (2023)

## Abstract (arXiv)

Centralized Training with Decentralized Execution (CTDE) is the dominant paradigm in multi-agent reinforcement learning (MARL), enabling agents to act independently at test time while leveraging additional information during training. However, the most prominent methods within CTDE, based on value decomposition, are limited in learning efficiency and final performance by partial observability in both training and execution. To overcome this limitation, in this work, we propose the framework of tutoring: In training, the agents share information in their latent space to develop well-informed policies that achieve strong performance. Then, to recover decentralized execution, these policies concurrently adjust to anticipate lack of communication, and they are distilled into counterparts that rely solely on local observations. We demonstrate the effectiveness of our approach on Hallway, which, to the best of our knowledge, has not been solved before without test-time communication, SMAC under settings more difficult than the standard ones, and SMACv2.