# Mis-spoke or mis-lead: Achieving Robustness in Multi-Agent Communicative Reinforcement Learning

arXiv: https://arxiv.org/abs/2108.03803 · HTML: https://mlsanity.com/paper/2108.03803/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 50 references that are in the map.
- First posted on arXiv 9 Aug 2021.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [Ensemble Adversarial Training: Attacks and Defenses](https://mlsanity.com/paper/1705.07204/) (2017)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [The StarCraft Multi-Agent Challenge](https://mlsanity.com/paper/1902.04043/) (2019)
- [Learning to Communicate with Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1605.06676/) (2016)
- [Learning Multiagent Communication with Backpropagation](https://mlsanity.com/paper/1605.07736/) (2016)
- [QPLEX: Duplex Dueling Multi-Agent Q-Learning](https://mlsanity.com/paper/2008.01062/) (2020)
- [A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning](https://mlsanity.com/paper/1711.00832/) (2017)
- [Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observations](https://mlsanity.com/paper/2003.08938/) (2020)
- [Adversarial Policies: Attacking Deep Reinforcement Learning](https://mlsanity.com/paper/1905.10615/) (2019)
- [TarMAC: Targeted Multi-Agent Communication](https://mlsanity.com/paper/1810.11187/) (2018)
- [Learning Attentional Communication for Multi-Agent Cooperation](https://mlsanity.com/paper/1805.07733/) (2018)
- [Tactics of Adversarial Attack on Deep Reinforcement Learning Agents](https://mlsanity.com/paper/1703.06748/) (2017)
- [Robust Reinforcement Learning on State Observations with Learned Optimal Adversary](https://mlsanity.com/paper/2101.08452/) (2021)
- [Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks](https://mlsanity.com/paper/1812.09755/) (2018)
- [Building Generalizable Agents with a Realistic and Rich 3D Environment](https://mlsanity.com/paper/1801.02209/) (2018)
- [Deep Coordination Graphs](https://mlsanity.com/paper/1910.00091/) (2019)
- [Off-Policy Multi-Agent Decomposed Policy Gradients](https://mlsanity.com/paper/2007.12322/) (2020)
- [Learning to Schedule Communication in Multi-agent Reinforcement Learning](https://mlsanity.com/paper/1902.01554/) (2019)
- [Learning Nearly Decomposable Value Functions Via Communication Minimization](https://mlsanity.com/paper/1910.05366/) (2019)
- [Learning Efficient Multi-agent Communication: An Information Bottleneck Approach](https://mlsanity.com/paper/1911.06992/) (2019)
- [A Generalized Training Approach for Multiagent Learning](https://mlsanity.com/paper/1909.12823/) (2019)
- [Adversarial Attacks On Multi-Agent Communication](https://mlsanity.com/paper/2101.06560/) (2021)

## Built on by (most cited first, 18 of 18)

- [A Survey of Multi-Agent Deep Reinforcement Learning with Communication](https://mlsanity.com/paper/2203.08975/) (2022)
- [Attacking Cooperative Multi-Agent Reinforcement Learning by Adversarial Minority Influence](https://mlsanity.com/paper/2302.03322/) (2023)
- [Robust multi-agent coordination via evolutionary generation of auxiliary adversarial attackers](https://mlsanity.com/paper/2305.05909/) (2023)
- [Heterogeneous Multi-agent Zero-Shot Coordination by Coevolution](https://mlsanity.com/paper/2208.04957/) (2022)
- [Certifiably Robust Policy Learning against Adversarial Communication in Multi-agent Systems](https://mlsanity.com/paper/2206.10158/) (2022)
- [Rethinking Adversarial Policies: A Generalized Attack Formulation and Provable Defense in RL](https://mlsanity.com/paper/2305.17342/) (2023)
- [Trust-based Consensus in Multi-Agent Reinforcement Learning Systems](https://mlsanity.com/paper/2205.12880/) (2022)
- [Communication-Robust Multi-Agent Learning by Adaptable Auxiliary Multi-Agent Adversary Generation](https://mlsanity.com/paper/2305.05116/) (2023)
- [Robust Multi-Agent Reinforcement Learning by Mutual Information Regularization](https://mlsanity.com/paper/2310.09833/) (2023)
- [Wolfpack Adversarial Attack for Robust Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2502.02844/) (2025)
- [Robust Communicative Multi-Agent Reinforcement Learning with Active Defense](https://mlsanity.com/paper/2312.11545/) (2023)
- [SoK: Adversarial Machine Learning Attacks and Defences in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2301.04299/) (2023)
- [Robust Multi-agent Communication via Multi-view Message Certification](https://mlsanity.com/paper/2305.13936/) (2023)
- [Efficient Communication via Self-supervised Information Aggregation for Online and Offline Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2302.09605/) (2023)
- [Interaction-Breaking Adversarial Learning Framework for Robust Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2605.18024/) (2026)
- [Robust Event-Driven Interactions in Cooperative Multi-Agent Learning](https://mlsanity.com/paper/2204.03361/) (2022)
- [Communication Gain and Delay Cost Under Cross-Timestep Delays in Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2604.03785/) (2026)
- [Finding the Weakest Link: Adversarial Attack against Multi-Agent Communications](https://mlsanity.com/paper/2605.13170/) (2026)

## Abstract (arXiv)

Recent studies in multi-agent communicative reinforcement learning (MACRL) have demonstrated that multi-agent coordination can be greatly improved by allowing communication between agents. Meanwhile, adversarial machine learning (ML) has shown that ML models are vulnerable to attacks. Despite the increasing concern about the robustness of ML algorithms, how to achieve robust communication in multi-agent reinforcement learning has been largely neglected. In this paper, we systematically explore the problem of adversarial communication in MACRL. Our main contributions are threefold. First, we propose an effective method to perform attacks in MACRL, by learning a model to generate optimal malicious messages. Second, we develop a defence method based on message reconstruction, to maintain multi-agent coordination under message attacks. Third, we formulate the adversarial communication problem as a two-player zero-sum game and propose a game-theoretical method R-MACRL to improve the worst-case defending performance. Empirical results demonstrate that many state-of-the-art MACRL methods are vulnerable to message attacks, and our method can significantly improve their robustness.