mlsanityData to 14 Oct 2026

Deep Reinforcement Learning Policies Learn Shared Adversarial Features Across MDPs

arXiv 2112.09025 ↗First posted 16 Dec 2021cs.LG cs.AI stat.MLcited by 12

Not on the map. Why?

Abstract (arXiv)

The use of deep neural networks as function approximators has led to striking progress for reinforcement learning algorithms and applications. Yet the knowledge we have on decision boundary geometry and the loss landscape of neural policies is still quite limited. In this paper we propose a framework to investigate the decision boundary and loss landscape similarities across states and across MDPs. We conduct experiments in various games from Arcade Learning Environment, and discover that high sensitivity directions for neural policies are correlated across MDPs. We argue that these high sensitivity directions support the hypothesis that non-robust features are shared across training environments of reinforcement learning agents. We believe our results reveal fundamental properties of the environments used in deep reinforcement learning training, and represent a tangible step towards building robust and reliable deep reinforcement learning agents.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 31 of its 42 references that are in the map. First posted on arXiv 16 Dec 2021.

Builds on

31 of 42 references.

References · 31

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  4. 2013Intriguing properties of neural networkscited by 5,155
  5. 2015Continuous control with deep reinforcement learningcited by 3,407
  6. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  7. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  8. 2016Adversarial examples in the physical worldcited by 1,769
  9. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  10. 2016OpenAI Gymcited by 1,736
  11. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
  12. 2015Prioritized Experience Replaycited by 1,099
19 more
  1. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 953
  2. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  3. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
  4. 2020On Adaptive Attacks to Adversarial Example Defensescited by 555
  5. 2017Distributional Reinforcement Learning with Quantile Regressioncited by 474
  6. 2017Robust Adversarial Reinforcement Learningcited by 451
  7. 2017Adversarial Attacks on Neural Network Policiescited by 320
  8. 2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 238
  9. 2019Adversarial Policies: Attacking Deep Reinforcement Learningcited by 214
  10. 2020Revisiting Fundamentals of Experience Replaycited by 175
  11. 2017Tactics of Adversarial Attack on Deep Reinforcement Learning Agentscited by 166
  12. 2017EAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examplescited by 155
  13. 2017Robust Deep Reinforcement Learning with Adversarial Attackscited by 154
  14. 2017Delving into adversarial attacks on deep policiescited by 104
  15. 2019Statistics and Samples in Distributional Reinforcement Learningcited by 63
  16. 2020Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learningcited by 63
  17. 2020Meta-Gradient Reinforcement Learning with an Objective Discovered Onlinecited by 52
  18. 2019Off-Policy Actor-Critic with Shared Experience Replaycited by 39
  19. 2021Investigating Vulnerabilities of Deep Neural Policiescited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 3; 2024: 3; 2025: 1; 2026: 3.

12 papers.

  1. 2022On the Robustness of Safe Reinforcement Learning under Observational Perturbationscited by 38
  2. 2023Adversarial Robust Deep Reinforcement Learning Requires Redefining Robustnesscited by 19
  3. 2024A Survey Analyzing Generalization in Deep Reinforcement Learningcited by 8
  4. 2023Detecting Adversarial Directions in Deep Reinforcement Learning to Make Robust Decisionscited by 7
  5. 2024Understanding and Diagnosing Deep Reinforcement Learningcited by 6
  6. 2022Targeted Adversarial Attacks on Deep Reinforcement Learning Policies via Model Checkingcited by 5
  7. 2024Benchmarking Deep Reinforcement Learning for Navigation in Denied Sensor Environmentscited by 4
  8. 2026Safety, Security, and Cognitive Risks in World Modelscited by 3
  9. 2023FLARE: Fingerprinting Deep Reinforcement Learning Agents using Universal Adversarial Maskscited by 2
  10. 2025Microservice Deployment in Space Computing Power Networks via Robust Reinforcement Learningcited by 1
  11. 2026Fuz-RL: A Fuzzy-Guided Robust Framework for Safe Reinforcement Learning under Uncertaintycited by 0
  12. 2026Boosting Transferable Adversarial Attacks against Deep Reinforcement Learningcited by 0