# Investigating Vulnerabilities of Deep Neural Policies

arXiv: https://arxiv.org/abs/2108.13093 · HTML: https://mlsanity.com/paper/2108.13093/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 29 references that are in the map.
- First posted on arXiv 30 Aug 2021.

## Builds on (most cited first, 19 of 19)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples](https://mlsanity.com/paper/1802.00420/) (2018)
- [Dueling Network Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06581/) (2015)
- [QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation](https://mlsanity.com/paper/1806.10293/) (2018)
- [Robust Adversarial Reinforcement Learning](https://mlsanity.com/paper/1703.02702/) (2017)
- [Deep Speech: Scaling up end-to-end speech recognition](https://mlsanity.com/paper/1412.5567/) (2014)
- [Adversarial Attacks on Neural Network Policies](https://mlsanity.com/paper/1702.02284/) (2017)
- [Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observations](https://mlsanity.com/paper/2003.08938/) (2020)
- [Adversarial Policies: Attacking Deep Reinforcement Learning](https://mlsanity.com/paper/1905.10615/) (2019)
- [Delving into adversarial attacks on deep policies](https://mlsanity.com/paper/1705.06452/) (2017)
- [On the Hardness of Robust Classification](https://mlsanity.com/paper/1909.05822/) (2019)
- [Deep Reinforcement Learning for Surgical Gesture Segmentation and Classification](https://mlsanity.com/paper/1806.08089/) (2018)
- [Empirically Measuring Concentration: Fundamental Limits on Intrinsic Robustness](https://mlsanity.com/paper/1905.12202/) (2019)

## Built on by (most cited first, 15 of 15)

- [Efficient Adversarial Training without Attacking: Worst-Case-Aware Robust Reinforcement Learning](https://mlsanity.com/paper/2210.05927/) (2022)
- [Adversarial Robust Deep Reinforcement Learning Requires Redefining Robustness](https://mlsanity.com/paper/2301.07487/) (2023)
- [Deep Reinforcement Learning Policies Learn Shared Adversarial Features Across MDPs](https://mlsanity.com/paper/2112.09025/) (2021)
- [A Survey Analyzing Generalization in Deep Reinforcement Learning](https://mlsanity.com/paper/2401.02349/) (2024)
- [Explaining RL Decisions with Trajectories](https://mlsanity.com/paper/2305.04073/) (2023)
- [Detecting Adversarial Directions in Deep Reinforcement Learning to Make Robust Decisions](https://mlsanity.com/paper/2306.05873/) (2023)
- [Improve Robustness of Reinforcement Learning against Observation Perturbations via l_∞ Lipschitz Policy Networks](https://mlsanity.com/paper/2312.08751/) (2023)
- [Understanding and Diagnosing Deep Reinforcement Learning](https://mlsanity.com/paper/2406.16979/) (2024)
- [Targeted Adversarial Attacks on Deep Reinforcement Learning Policies via Model Checking](https://mlsanity.com/paper/2212.05337/) (2022)
- [SoK: Adversarial Machine Learning Attacks and Defences in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2301.04299/) (2023)
- [Robust Deep Reinforcement Learning against Adversarial Behavior Manipulation](https://mlsanity.com/paper/2406.03862/) (2024)
- [Red Teaming with Mind Reading: White-Box Adversarial Policies Against RL Agents](https://mlsanity.com/paper/2209.02167/) (2022)
- ['Explaining RL Decisions with Trajectories': A Reproducibility Study](https://mlsanity.com/paper/2411.07200/) (2024)
- [Microservice Deployment in Space Computing Power Networks via Robust Reinforcement Learning](https://mlsanity.com/paper/2501.06244/) (2025)
- [Game-Theoretic Robust Reinforcement Learning Handles Temporally-Coupled Perturbations](https://mlsanity.com/paper/2307.12062/) (2023)

## Abstract (arXiv)

Reinforcement learning policies based on deep neural networks are vulnerable to imperceptible adversarial perturbations to their inputs, in much the same way as neural network image classifiers. Recent work has proposed several methods to improve the robustness of deep reinforcement learning agents to adversarial perturbations based on training in the presence of these imperceptible perturbations (i.e. adversarial training). In this paper, we study the effects of adversarial training on the neural policy learned by the agent. In particular, we follow two distinct parallel approaches to investigate the outcomes of adversarial training on deep neural policies based on worst-case distributional shift and feature sensitivity. For the first approach, we compare the Fourier spectrum of minimal perturbations computed for both adversarially trained and vanilla trained neural policies. Via experiments in the OpenAI Atari environments we show that minimal perturbations computed for adversarially trained policies are more focused on lower frequencies in the Fourier domain, indicating a higher sensitivity of these policies to low frequency perturbations. For the second approach, we propose a novel method to measure the feature sensitivities of deep neural policies and we compare these feature sensitivity differences in state-of-the-art adversarially trained deep neural policies and vanilla trained deep neural policies. We believe our results can be an initial step towards understanding the relationship between adversarial training and different notions of robustness for neural policies.