Data to 14 Oct 2026

Adversarial Attacks on Neural Network Policies

arXiv 1702.02284 ↗First posted 8 Feb 2017cs.LG cs.CR stat.MLcited by 320

Not on the map. Why?

Cited by 64 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Machine learning classifiers are known to be vulnerable to inputs maliciously constructed by adversaries to force misclassification. Such adversarial examples have been extensively studied in the context of computer vision applications. In this work, we show adversarial attacks are also effective when targeting neural network policies in reinforcement learning. Specifically, we show existing adversarial example crafting techniques can be used to significantly degrade test-time performance of trained policies. Our threat model considers adversaries capable of introducing small perturbations to the raw input of the policy. We characterize the degree of vulnerability across tasks and training algorithms, for a subclass of adversarial-example attacks in white-box and black-box settings. Regardless of the learned task or training algorithm, we observe a significant drop in performance, even with small adversarial perturbations that do not interfere with human perception. Videos are available at http://rll.berkeley.edu/adversarial.

Key facts. Cited by 320 papers in the map (data through 14 Oct 2026). First posted on arXiv 8 Feb 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 1; 2019: 1; 2020: 0; 2021: 62; 2022: 64; 2023: 58; 2024: 49; 2025: 48; 2026: 37.

320 papers.

Sort:

By area:

2018Scalable agent alignment via reward modeling: a research directioncited by 447
2021Recent Advances in Adversarial Training for Adversarial Robustnesscited by 279
2021Robust Reinforcement Learning on State Observations with Learned Optimal Adversarycited by 135
2021Advances in adversarial attacks and defenses in computer vision: A surveycited by 127
2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 122
2022RAMBO-RL: Robust Adversarial Model-Based Offline Reinforcement Learningcited by 115
2021Online Robust Reinforcement Learning with Model Uncertaintycited by 96
2022RORL: Robust Offline Reinforcement Learning via Conservative Smoothingcited by 84
2022Toward Next-Generation Artificial Intelligence: Catalyzing the NeuroAI Revolutioncited by 78
2021A Survey on Interpretable Reinforcement Learningcited by 73
2019Policy Poisoning in Batch Reinforcement Learning and Controlcited by 70
2022Policy Gradient Method For Robust Reinforcement Learningcited by 69

38 more

2021BACKDOORL: Backdoor Attack against Competitive Reinforcement Learningcited by 68
2021TDGIA:Effective Injection Attacks on Graph Neural Networkscited by 67
2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluationscited by 67
2022SafeBench: A Benchmarking Platform for Safety Evaluation of Autonomous Vehiclescited by 66
2021Robot Learning from Randomized Simulations: A Reviewcited by 63
2021Who Is the Strongest Enemy? Towards Optimal and Efficient Evasion Attacks in Deep RLcited by 55
2021Adversarial Attacks On Multi-Agent Communicationcited by 49
2022Sound Adversarial Audio-Visual Navigationcited by 48
2022Uncertainty Quantification of Collaborative Detection for Self-Drivingcited by 46
2021Adversarial Attacks for Tabular Data: Application to Fraud Detection and Imbalanced Datacited by 45
2024Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignmentcited by 44
2022Efficient Adversarial Training without Attacking: Worst-Case-Aware Robust Reinforcement Learningcited by 43
2021CROP: Certifying Robust Policies for Reinforcement Learning through Functional Smoothingcited by 42
2022Towards Formal XAI: Formally Approximate Minimal Explanations of Neural Networkscited by 42
2022Towards Safe Reinforcement Learning via Constraining Conditional Value-at-Riskcited by 41
2021Policy Smoothing for Provably Robust Reinforcement Learningcited by 40
2021Adversarial Robustness with Semi-Infinite Constrained Learningcited by 39
2022On the Robustness of Safe Reinforcement Learning under Observational Perturbationscited by 38
2022Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizabilitycited by 36
2023Robust Multi-Agent Reinforcement Learning with State Uncertaintycited by 36
2021Evading Adversarial Example Detection Defenses with Orthogonal Projected Gradient Descentcited by 35
2023DeceptPrompt: Exploiting LLM-driven Code Generation via Adversarial Natural Language Instructionscited by 33
2022Distributionally Robust Offline Reinforcement Learning with Linear Function Approximationcited by 32
2024Attacks and Defenses for Generative Diffusion Models: A Comprehensive Surveycited by 32
20213DB: A Framework for Debugging Computer Vision Modelscited by 31
2025SIMA 2: A Generalist Embodied Agent for Virtual Worldscited by 31
2021Deep Learning-Based Autonomous Driving Systems: A Survey of Attacks and Defensescited by 30
2021Towards Distraction-Robust Active Visual Trackingcited by 30
2021Robust Predictable Controlcited by 29
2021Provably Efficient Black-Box Action Poisoning Attacks Against Reinforcement Learningcited by 29
2022Adversarial Policies Beat Superhuman Go AIscited by 29
2022Fooling the Eyes of Autonomous Vehicles: Robust Physical Adversarial Examples Against Traffic Sign Recognition Systemscited by 27
2023Interpretability Benchmark for Evaluating Spatial Misalignment of Prototypical Parts Explanationscited by 27
2021Reward Poisoning in Reinforcement Learning: Attacks Against Unknown Learners in Unknown Environmentscited by 26
2022COPA: Certifying Robust Policies for Offline Reinforcement Learning against Poisoning Attackscited by 26
2022BAFFLE: Hiding Backdoors in Offline Reinforcement Learning Datasetscited by 25
2025RobustVLA: On Robustness of Vision-Language-Action Model against Multi-Modal Perturbationscited by 25
2024Adversarial Attacks of Vision Tasks in the Past 10 Years: A Surveycited by 23

50 most cited of 320 All 320 (JSON)