# Sparse Adversarial Attack in Multi-agent Reinforcement Learning

arXiv: https://arxiv.org/abs/2205.09362 · HTML: https://mlsanity.com/paper/2205.09362/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 38 references that are in the map.
- First posted on arXiv 19 May 2022.

## Builds on (most cited first, 27 of 27)

- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Counterfactual Multi-Agent Policy Gradients](https://mlsanity.com/paper/1705.08926/) (2017)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [The StarCraft Multi-Agent Challenge](https://mlsanity.com/paper/1902.04043/) (2019)
- [Value-Decomposition Networks For Cooperative Multi-Agent Learning](https://mlsanity.com/paper/1706.05296/) (2017)
- [QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1905.05408/) (2019)
- [Adversarial Attacks on Neural Network Policies](https://mlsanity.com/paper/1702.02284/) (2017)
- [Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/1610.03295/) (2016)
- [Weighted QMIX: Expanding Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2006.10800/) (2020)
- [Tactics of Adversarial Attack on Deep Reinforcement Learning Agents](https://mlsanity.com/paper/1703.06748/) (2017)
- [ROMA: Multi-Agent Reinforcement Learning with Emergent Roles](https://mlsanity.com/paper/2003.08039/) (2020)
- [Vulnerability of Deep Reinforcement Learning to Policy Induction Attacks](https://mlsanity.com/paper/1701.04143/) (2017)
- [Robust Reinforcement Learning for Continuous Control with Model Misspecification](https://mlsanity.com/paper/1906.07516/) (2019)
- [Multi-Agent Deep Reinforcement Learning for Dynamic Power Allocation in Wireless Networks](https://mlsanity.com/paper/1808.00490/) (2018)
- [Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learning](https://mlsanity.com/paper/2005.07099/) (2020)
- [Wasserstein Robust Reinforcement Learning](https://mlsanity.com/paper/1907.13196/) (2019)
- [On the Robustness of Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2003.03722/) (2020)
- [Characterizing Attacks on Deep Reinforcement Learning](https://mlsanity.com/paper/1907.09470/) (2019)
- [Spatiotemporally Constrained Action Space Attacks on Deep Reinforcement Learning Agents](https://mlsanity.com/paper/1909.02583/) (2019)
- [Optimal Attacks on Reinforcement Learning Policies](https://mlsanity.com/paper/1907.13548/) (2019)
- [Towards Comprehensive Testing on the Robustness of Cooperative Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2204.07932/) (2022)
- [ROMAX: Certifiably Robust Deep Multiagent Reinforcement Learning via Convex Relaxation](https://mlsanity.com/paper/2109.06795/) (2021)
- [Sequential Attacks on Agents for Long-Term Adversarial Goals](https://mlsanity.com/paper/1805.12487/) (2018)
- [Robust Constrained Reinforcement Learning for Continuous Control with Model Misspecification](https://mlsanity.com/paper/2010.10644/) (2020)
- [Adversarial Exploitation of Policy Imitation](https://mlsanity.com/paper/1906.01121/) (2019)
- [Enhanced Adversarial Strategically-Timed Attacks against Deep Reinforcement Learning](https://mlsanity.com/paper/2002.09027/) (2020)
- [Robust Reinforcement Learning under model misspecification](https://mlsanity.com/paper/2103.15370/) (2021)

## Built on by (most cited first, 8 of 8)

- [Robust multi-agent coordination via evolutionary generation of auxiliary adversarial attackers](https://mlsanity.com/paper/2305.05909/) (2023)
- [Communication-Robust Multi-Agent Learning by Adaptable Auxiliary Multi-Agent Adversary Generation](https://mlsanity.com/paper/2305.05116/) (2023)
- [Multi-Agent Reinforcement Learning: Methods, Applications, Visionary Prospects, and Challenges](https://mlsanity.com/paper/2305.10091/) (2023)
- [Robust Multi-agent Communication via Multi-view Message Certification](https://mlsanity.com/paper/2305.13936/) (2023)
- [Enhancing the Robustness of QMIX against State-adversarial Attacks](https://mlsanity.com/paper/2307.00907/) (2023)
- [Constrained Black-Box Attacks Against Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2508.09275/) (2025)
- [Detection of adversarial intent in Human-AI teams using LLMs](https://mlsanity.com/paper/2603.20976/) (2026)
- [When Actions Disappear: Adversarial Action Removal in Self-Play Reinforcement Learning](https://mlsanity.com/paper/2605.16312/) (2026)

## Abstract (arXiv)

Cooperative multi-agent reinforcement learning (cMARL) has many real applications, but the policy trained by existing cMARL algorithms is not robust enough when deployed. There exist also many methods about adversarial attacks on the RL system, which implies that the RL system can suffer from adversarial attacks, but most of them focused on single agent RL. In this paper, we propose a textitsparse adversarial attack on cMARL systems. We use (MA)RL with regularization to train the attack policy. Our experiments show that the policy trained by the current cMARL algorithm can obtain poor performance when only one or a few agents in the team (e.g., 1 of 8 or 5 of 25) were attacked at a few timesteps (e.g., attack 3 of total 40 timesteps).