mlsanityData to 7 Oct 2026

SoK: Adversarial Machine Learning Attacks and Defences in Multi-Agent Reinforcement Learning

arXiv 2301.04299 ↗First posted 11 Jan 2023cs.LG cs.AI cs.CRcited by 3

Not on the map. Why?

Abstract (arXiv)

Multi-Agent Reinforcement Learning (MARL) is vulnerable to Adversarial Machine Learning (AML) attacks and needs adequate defences before it can be used in real world applications. We have conducted a survey into the use of execution-time AML attacks against MARL and the defences against those attacks. We surveyed related work in the application of AML in Deep Reinforcement Learning (DRL) and Multi-Agent Learning (MAL) to inform our analysis of AML for MARL. We propose a novel perspective to understand the manner of perpetrating an AML attack, by defining Attack Vectors. We develop two new frameworks to address a gap in current modelling frameworks, focusing on the means and tempo of an AML attack against MARL, and identify knowledge gaps and future avenues of research.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 73 of its 121 references that are in the map. First posted on arXiv 11 Jan 2023.

Builds on

73 of 121 references.

References · 73

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2015Continuous control with deep reinforcement learningcited by 3,279
  6. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  7. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  8. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  9. 2015The Limitations of Deep Learning in Adversarial Settingscited by 784
  10. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  11. 2017One pixel attack for fooling deep neural networkscited by 579
  12. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 561
38 more
  1. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 542
  2. 2017Countering Adversarial Images using Input Transformationscited by 517
  3. 2017Adversarial Patchcited by 514
  4. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 457
  5. 2017Robust Adversarial Reinforcement Learningcited by 439
  6. 2016Learning Multiagent Communication with Backpropagationcited by 355
  7. 2017Adversarial Attacks on Neural Network Policiescited by 314
  8. 2021Recent Advances in Adversarial Training for Adversarial Robustnesscited by 276
  9. 2018Actor-Attention-Critic for Multi-Agent Reinforcement Learningcited by 259
  10. 2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 235
  11. 2013The Complexity of Decentralized Control of Markov Decision Processescited by 225
  12. 2018Adversarial Risk and the Dangers of Evaluating Against Weak Attackscited by 221
  13. 2019Adversarial Policies: Attacking Deep Reinforcement Learningcited by 209
  14. 2018TarMAC: Targeted Multi-Agent Communicationcited by 201
  15. 2018Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agentscited by 189
  16. 2018Deep Reinforcement Learning and the Deadly Triadcited by 170
  17. 2019Action Robust Reinforcement Learning and Applications in Continuous Controlcited by 167
  18. 2017Tactics of Adversarial Attack on Deep Reinforcement Learning Agentscited by 163
  19. 2017Robust Deep Reinforcement Learning with Adversarial Attackscited by 152
  20. 2020Adversarial Machine Learning -- Industry Perspectivescited by 121
  21. 2018Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Taskscited by 114
  22. 2017Delving into adversarial attacks on deep policiescited by 104
  23. 2022A Survey of Multi-Agent Deep Reinforcement Learning with Communicationcited by 103
  24. 2017Vulnerability of Deep Reinforcement Learning to Policy Induction Attackscited by 91
  25. 2019Learning Nearly Decomposable Value Functions Via Communication Minimizationcited by 85
  26. 2020Robust Deep Reinforcement Learning through Adversarial Losscited by 73
  27. 2019Probabilistic Recursive Reasoning for Multi-Agent Reinforcement Learningcited by 68
  28. 2020Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learningcited by 62
  29. 2020Challenges and Countermeasures for Adversarial Attacks on Deep Reinforcement Learningcited by 61
  30. 2021CybORG: A Gym for the Development of Autonomous Cyber Agentscited by 58
  31. 2021Who Is the Strongest Enemy? Towards Optimal and Efficient Evasion Attacks in Deep RLcited by 54
  32. 2021Adversarial Attacks On Multi-Agent Communicationcited by 48
  33. 2021A Deep Value-network Based Approach for Multi-Driver Order Dispatchingcited by 45
  34. 2021CROP: Certifying Robust Policies for Reinforcement Learning through Functional Smoothingcited by 41
  35. 2022Towards Safe Reinforcement Learning via Constraining Conditional Value-at-Riskcited by 40
  36. 2020On the Robustness of Cooperative Multi-Agent Reinforcement Learningcited by 39
  37. 2021Discrete-Valued Neural Communicationcited by 39
  38. 2019Non-Stationary Markov Decision Processes, a Worst-Case Approach using Model-Based Reinforcement Learning, Extended versioncited by 36

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 1; 2025: 1.

3 papers.

  1. 2023Learning Cyber Defence Tactics from Scratch with Multi-Agent Reinforcement Learningcited by 12
  2. 2024A Novel Bifurcation Method for Observation Perturbation Attacks on Reinforcement Learning Agents: Load Altering Attacks on a Cyber Physical Power Systemcited by 0
  3. 2025Collapsing Sequence-Level Data-Policy Coverage via Poisoning Attack in Offline Reinforcement Learningcited by 0