mlsanityData to 7 Oct 2026

Who Is the Strongest Enemy? Towards Optimal and Efficient Evasion Attacks in Deep RL

arXiv 2106.05087 ↗First posted 9 Jun 2021cs.LG cs.CRcited by 54

Not on the map. Why?

Abstract (arXiv)

Evaluating the worst-case performance of a reinforcement learning (RL) agent under the strongest/optimal adversarial perturbations on state observations (within some constraints) is crucial for understanding the robustness of RL agents. However, finding the optimal adversary is challenging, in terms of both whether we can find the optimal attack and how efficiently we can find it. Existing works on adversarial RL either use heuristics-based methods that may not find the strongest adversary, or directly train an RL-based adversary by treating the agent as a part of the environment, which can find the optimal adversary but may become intractable in a large state space. This paper introduces a novel attacking method to find the optimal attacks through collaboration between a designed function named "actor" and an RL-based learner named "director". The actor crafts state perturbations for a given policy perturbation direction, and the director learns to propose the best policy perturbation directions. Our proposed algorithm, PA-AD, is theoretically optimal and significantly more efficient than prior RL-based works in environments with large state spaces. Empirical results show that our proposed PA-AD universally outperforms state-of-the-art attacking methods in various Atari and MuJoCo environments. By applying PA-AD to adversarial training, we achieve state-of-the-art empirical robustness in multiple tasks under strong adversaries. The codebase is released at https://github.com/umd-huang-lab/paad_adv_rl.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). Builds on 30 of its 52 references that are in the map. First posted on arXiv 9 Jun 2021.

Builds on

30 of 52 references.

References · 30

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  5. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  6. 2015Prioritized Experience Replaycited by 1,070
  7. 2017Robust Adversarial Reinforcement Learningcited by 439
  8. 2017Adversarial Attacks on Neural Network Policiescited by 314
  9. 2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 235
  10. 2019Adversarial Policies: Attacking Deep Reinforcement Learningcited by 209
  11. 2019Action Robust Reinforcement Learning and Applications in Continuous Controlcited by 167
  12. 2017Tactics of Adversarial Attack on Deep Reinforcement Learning Agentscited by 163
18 more
  1. 2017Robust Deep Reinforcement Learning with Adversarial Attackscited by 152
  2. 2021Robust Reinforcement Learning on State Observations with Learned Optimal Adversarycited by 133
  3. 2017Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximationcited by 119
  4. 2017Delving into adversarial attacks on deep policiescited by 104
  5. 2017Vulnerability of Deep Reinforcement Learning to Policy Induction Attackscited by 91
  6. 2020Adaptive Reward-Poisoning Attacks against Reinforcement Learningcited by 86
  7. 2020Policy Teaching via Environment Poisoning: Training-time Adversarial Attacks against Reinforcement Learningcited by 81
  8. 2020Robust Deep Reinforcement Learning through Adversarial Losscited by 73
  9. 2019Deceptive Reinforcement Learning Under Adversarial Manipulations on Cost Signalscited by 50
  10. 2019Certified Adversarial Robustness for Deep Reinforcement Learningcited by 47
  11. 2020Vulnerability-Aware Poisoning Mechanism for Online RL with Unknown Dynamicscited by 38
  12. 2019Characterizing Attacks on Deep Reinforcement Learningcited by 31
  13. 2020Robustifying Reinforcement Learning Agents via Action Space Adversarial Trainingcited by 30
  14. 2019The Value Function Polytope in Reinforcement Learningcited by 29
  15. 2019Online Robustness Training for Deep Reinforcement Learningcited by 26
  16. 2019Optimal Attacks on Reinforcement Learning Policiescited by 21
  17. 2019Snooping Attacks on Deep Reinforcement Learningcited by 17
  18. 2020Query-based Targeted Action-Space Adversarial Policies on Deep Reinforcement Learning Agentscited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 8; 2023: 17; 2024: 12; 2025: 9; 2026: 6.

54 papers.

Sort:

By area:

  1. 2022Efficient Adversarial Training without Attacking: Worst-Case-Aware Robust Reinforcement Learningcited by 42
  2. 2021CROP: Certifying Robust Policies for Reinforcement Learning through Functional Smoothingcited by 41
  3. 2022On the Robustness of Safe Reinforcement Learning under Observational Perturbationscited by 38
  4. 2022Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizabilitycited by 32
  5. 2021Reinforcement Learning for Feedback-Enabled Cyber Resiliencecited by 18
  6. 2022Adversarial Cheap Talkcited by 16
  7. 2023Attacking Cooperative Multi-Agent Reinforcement Learning by Adversarial Minority Influencecited by 16
  8. 2024RAT: Adversarial Attacks on Deep Reinforcement Agents for Targeted Behaviorscited by 16
  9. 2023Robust multi-agent coordination via evolutionary generation of auxiliary adversarial attackerscited by 15
  10. 2022Illusory Attacks: Information-Theoretic Detectability Matters in Adversarial Attackscited by 12
  11. 2023Optimal Attack and Defense for Reinforcement Learningcited by 12
  12. 2023Adjustable Robust Reinforcement Learning for Online 3D Bin Packingcited by 9
38 more
  1. 2024Beyond Worst-case Attacks: Robust RL with Adaptive Defense via Non-dominated Policiescited by 8
  2. 2022Certifiably Robust Policy Learning against Adversarial Communication in Multi-agent Systemscited by 7
  3. 2023Rethinking Adversarial Policies: A Generalized Attack Formulation and Provable Defense in RLcited by 7
  4. 2023Robustness Testing for Multi-Agent Reinforcement Learning: State Perturbations on Critical Agentscited by 7
  5. 2023Improve Robustness of Reinforcement Learning against Observation Perturbations via l_∞ Lipschitz Policy Networkscited by 7
  6. 2022Distributional Reward Estimation for Effective Multi-Agent Deep Reinforcement Learningcited by 6
  7. 2023Multi-Agent Reinforcement Learning: Methods, Applications, Visionary Prospects, and Challengescited by 5
  8. 2024Towards Robust Policy: Enhancing Offline Reinforcement Learning with Adversarial Attacks and Defensescited by 5
  9. 2024Towards Optimal Adversarial Robust Q-learning with Bellman Infinity-errorcited by 4
  10. 2024Breaking the Barrier: Enhanced Utility and Robustness in Smoothed DRL Agentscited by 4
  11. 2024Mitigating Adversarial Perturbations for Deep Reinforcement Learning via Vector Quantizationcited by 4
  12. 2023SoK: Adversarial Machine Learning Attacks and Defences in Multi-Agent Reinforcement Learningcited by 3
  13. 2023Regret-Based Defense in Adversarial Reinforcement Learningcited by 3
  14. 2023Toward Evaluating Robustness of Reinforcement Learning with Adversarial Policycited by 3
  15. 2023Robust Multi-agent Communication via Multi-view Message Certificationcited by 3
  16. 2023Enhancing the Robustness of QMIX against State-adversarial Attackscited by 3
  17. 2023Certifiably Robust Reinforcement Learning through Model-Based Abstract Interpretationcited by 2
  18. 2023Efficient Action Robust Reinforcement Learning with Probabilistic Policy Execution Uncertaintycited by 2
  19. 2024Belief-Enriched Pessimistic Q-Learning against Adversarial State Perturbationscited by 2
  20. 2024Robust Deep Reinforcement Learning against Adversarial Behavior Manipulationcited by 2
  21. 2024On Minimizing Adversarial Counterfactual Error in Adversarial RLcited by 2
  22. 2022Security of Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 1
  23. 2023Optimal Cost Constrained Adversarial Attacks For Multiple Agent Systemscited by 1
  24. 2024Robustness Evaluation of Offline Reinforcement Learning for Robot Control Against Action Perturbationscited by 1
  25. 2025Action Robust Reinforcement Learning via Optimal Adversary Aware Policy Optimizationcited by 1
  26. 2025Empirical Study on Robustness and Resilience in Cooperative Multi-Agent Reinforcement Learningcited by 1
  27. 2023Game-Theoretic Robust Reinforcement Learning Handles Temporally-Coupled Perturbationscited by 0
  28. 2024On the Perturbed States for Transformed Input-robust Reinforcement Learningcited by 0
  29. 2024Robust off-policy Reinforcement Learning via Soft Constrained Adversarycited by 0
  30. 2025Collapsing Sequence-Level Data-Policy Coverage via Poisoning Attack in Offline Reinforcement Learningcited by 0
  31. 2025Off-Policy Actor-Critic for Adversarial Observation Robustness: Virtual Alternative Training via Symmetric Policy Evaluationcited by 0
  32. 2025Adversarial Reinforcement Learning Framework for ESP Cheater Simulationcited by 0
  33. 2025SAJA: A State-Action Joint Attack Framework on Multi-Agent Deep Reinforcement Learningcited by 0
  34. 2025Adversarial Fine-tuning in Offline-to-Online Reinforcement Learning for Robust Robot Controlcited by 0
  35. 2025Diffusion Guided Adversarial State Perturbations in Reinforcement Learningcited by 0
  36. 2025On the Tension Between Optimality and Adversarial Robustness in Policy Optimizationcited by 0
  37. 2026Advantage-based Temporal Attack in Reinforcement Learningcited by 0
  38. 2026Finding the Weakest Link: Adversarial Attack against Multi-Agent Communicationscited by 0

50 most cited of 54 All 54 (JSON)