mlsanityData to 7 Oct 2026

RoMFAC: A robust mean-field actor-critic reinforcement learning against adversarial perturbations on states

arXiv 2205.07229 ↗First posted 15 May 2022cs.LG cs.AIcited by 16

Not on the map. Why?

Abstract (arXiv)

Multi-agent deep reinforcement learning makes optimal decisions dependent on system states observed by agents, but any uncertainty on the observations may mislead agents to take wrong actions. The Mean-Field Actor-Critic reinforcement learning (MFAC) is well-known in the multi-agent field since it can effectively handle a scalability problem. However, it is sensitive to state perturbations that can significantly degrade the team rewards. This work proposes a Robust Mean-field Actor-Critic reinforcement learning (RoMFAC) that has two innovations: 1) a new objective function of training actors, composed of a policy gradient function that is related to the expected cumulative discount reward on sampled clean states and an action loss function that represents the difference between actions taken on clean and adversarial states; and 2) a repetitive regularization of the action loss, ensuring the trained actors to obtain excellent performance. Furthermore, this work proposes a game model named a State-Adversarial Stochastic Game (SASG). Despite the Nash equilibrium of SASG may not exist, adversarial perturbations to states in the RoMFAC are proven to be defensible based on SASG. Experimental results show that RoMFAC is robust against adversarial perturbations while maintaining its competitive performance in environments without perturbations.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 26 of its 62 references that are in the map. First posted on arXiv 15 May 2022.

Builds on

26 of 62 references.

References · 26

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  4. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  5. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  6. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 475
  7. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 388
  8. 2018On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Modelscited by 322
  9. 2017Adversarial Attacks on Neural Network Policiescited by 314
  10. 2020QPLEX: Duplex Dueling Multi-Agent Q-Learningcited by 267
  11. 2018Actor-Attention-Critic for Multi-Agent Reinforcement Learningcited by 259
  12. 2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 235
14 more
  1. 2018Mean Field Multi-Agent Reinforcement Learningcited by 202
  2. 2021Robust Reinforcement Learning on State Observations with Learned Optimal Adversarycited by 133
  3. 2021A Comprehensive Survey of Scene Graphs: Generation and Applicationcited by 125
  4. 2019Wasserstein Adversarial Examples via Projected Sinkhorn Iterationscited by 89
  5. 2020Robust Deep Reinforcement Learning through Adversarial Losscited by 73
  6. 2017MAgent: A Many-Agent Reinforcement Learning Platform for Artificial Collective Intelligencecited by 69
  7. 2020Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learningcited by 62
  8. 2020Challenges and Countermeasures for Adversarial Attacks on Deep Reinforcement Learningcited by 61
  9. 2020On the Robustness of Cooperative Multi-Agent Reinforcement Learningcited by 39
  10. 2020Stop-and-Go: Exploring Backdoor Attacks on Deep Reinforcement Learning-based Traffic Congestion Control Systemscited by 35
  11. 2020Q-value Path Decomposition for Deep Multiagent Reinforcement Learningcited by 23
  12. 2021ROMAX: Certifiably Robust Deep Multiagent Reinforcement Learning via Convex Relaxationcited by 17
  13. 2022Transferring Multi-Agent Reinforcement Learning Policies for Autonomous Driving using Sim-to-Realcited by 16
  14. 2017Sim-to-Real Transfer of Accurate Grasping with Eye-In-Hand Observations and Continuous Controlcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 6; 2024: 1; 2025: 5; 2026: 1.

16 papers.

Sort:

By area:

  1. 2023Robust Multi-Agent Reinforcement Learning via Adversarial Regularization: Theoretical Foundation and Stable Algorithmscited by 25
  2. 2022Asynchronous Distributed Bilevel Optimizationcited by 13
  3. 2023Safe Reinforcement Learning with Dual Robustnesscited by 10
  4. 2023Partially Observable Mean Field Multi-Agent Reinforcement Learning Based on Graph-Attentioncited by 7
  5. 2023Robustness Testing for Multi-Agent Reinforcement Learning: State Perturbations on Critical Agentscited by 7
  6. 2025Wolfpack Adversarial Attack for Robust Multi-Agent Reinforcement Learningcited by 6
  7. 2023Multi-Agent Reinforcement Learning: Methods, Applications, Visionary Prospects, and Challengescited by 5
  8. 2023Enhancing the Robustness of QMIX against State-adversarial Attackscited by 3
  9. 2025PNAct: Crafting Backdoor Attacks in Safe Reinforcement Learningcited by 2
  10. 2025Robust and Efficient Communication in Multi-Agent Reinforcement Learningcited by 2
  11. 2022Security of Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 1
  12. 2022Rollout Total Correlation for Deep Reinforcement Learningcited by 0
4 more
  1. 2024Off-OAB: Off-Policy Policy Gradient Method with Optimal Action-Dependent Baselinecited by 0
  2. 2025Policy Disruption in Reinforcement Learning:Adversarial Attack with Large Language Models and Critical State Identificationcited by 0
  3. 2025Pruning Cannot Hurt Robustness: Certified Trade-offs in Reinforcement Learningcited by 0
  4. 2026Diffusion Policy with Behavioral Advantage Correction for Offline Reinforcement Learningcited by 0