# Characterizing Attacks on Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1907.09470 · HTML: https://mlsanity.com/paper/1907.09470/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jul 2019.

## Built on by (most cited first, 30 of 31)

- [Robust Reinforcement Learning on State Observations with Learned Optimal Adversary](https://mlsanity.com/paper/2101.08452/) (2021)
- [Invisible for both Camera and LiDAR: Security of Multi-Sensor Fusion based Perception in Autonomous Driving Under Physical-World Attacks](https://mlsanity.com/paper/2106.09249/) (2021)
- [Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning](https://mlsanity.com/paper/2205.01992/) (2022)
- [Physical Attack on Monocular Depth Estimation with Optimal Adversarial Patches](https://mlsanity.com/paper/2207.04718/) (2022)
- [Who Is the Strongest Enemy? Towards Optimal and Efficient Evasion Attacks in Deep RL](https://mlsanity.com/paper/2106.05087/) (2021)
- [Efficient Adversarial Training without Attacking: Worst-Case-Aware Robust Reinforcement Learning](https://mlsanity.com/paper/2210.05927/) (2022)
- [Defending against Adversarial Audio via Diffusion Model](https://mlsanity.com/paper/2303.01507/) (2023)
- [What is the Solution for State-Adversarial Multi-Agent Reinforcement Learning?](https://mlsanity.com/paper/2212.02705/) (2022)
- [Are AlphaZero-like Agents Robust to Adversarial Perturbations?](https://mlsanity.com/paper/2211.03769/) (2022)
- [Understanding Adversarial Attacks on Observations in Deep Reinforcement Learning](https://mlsanity.com/paper/2106.15860/) (2021)
- [MDPFuzz: Testing Models Solving Markov Decision Processes](https://mlsanity.com/paper/2112.02807/) (2021)
- [Real-time Adversarial Perturbations against Deep Reinforcement Learning Policies: Attacks and Defenses](https://mlsanity.com/paper/2106.08746/) (2021)
- [COLEP: Certifiably Robust Learning-Reasoning Conformal Prediction via Probabilistic Circuits](https://mlsanity.com/paper/2403.11348/) (2024)
- [Sparse Adversarial Attack in Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2205.09362/) (2022)
- [Beyond Worst-case Attacks: Robust RL with Adaptive Defense via Non-dominated Policies](https://mlsanity.com/paper/2402.12673/) (2024)
- [Machine Learning for Security in Vehicular Networks: A Comprehensive Survey](https://mlsanity.com/paper/2105.15035/) (2021)
- [Network Defense is Not a Game](https://mlsanity.com/paper/2104.10262/) (2021)
- [Deep-Attack over the Deep Reinforcement Learning](https://mlsanity.com/paper/2205.00807/) (2022)
- [Adversarial joint attacks on legged robots](https://mlsanity.com/paper/2205.10098/) (2022)
- [Emerging Threats in Deep Learning-Based Autonomous Driving: A Comprehensive Survey](https://mlsanity.com/paper/2210.11237/) (2022)
- [Adversarial Body Shape Search for Legged Robots](https://mlsanity.com/paper/2205.10187/) (2022)
- [Imitating Opponent to Win: Adversarial Policy Imitation Learning in Two-player Competitive Games](https://mlsanity.com/paper/2210.16915/) (2022)
- [Security of Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2212.06123/) (2022)
- [FLARE: Fingerprinting Deep Reinforcement Learning Agents using Universal Adversarial Masks](https://mlsanity.com/paper/2307.14751/) (2023)
- [Adversarial Reinforcement Learning in Dynamic Channel Access and Power Control](https://mlsanity.com/paper/2105.05817/) (2021)
- [Provably Efficient Action-Manipulation Attack Against Continuous Reinforcement Learning](https://mlsanity.com/paper/2411.13116/) (2024)
- [Action Robust Reinforcement Learning via Optimal Adversary Aware Policy Optimization](https://mlsanity.com/paper/2507.03372/) (2025)
- [Efficient Preference Poisoning Attack on Offline RLHF](https://mlsanity.com/paper/2605.02495/) (2026)
- [Game-Theoretic Robust Reinforcement Learning Handles Temporally-Coupled Perturbations](https://mlsanity.com/paper/2307.12062/) (2023)
- [Towards Robust Deep Reinforcement Learning against Environmental State Perturbation](https://mlsanity.com/paper/2506.08961/) (2025)

## Abstract (arXiv)

Recent studies show that Deep Reinforcement Learning (DRL) models are vulnerable to adversarial attacks, which attack DRL models by adding small perturbations to the observations. However, some attacks assume full availability of the victim model, and some require a huge amount of computation, making them less feasible for real world applications. In this work, we make further explorations of the vulnerabilities of DRL by studying other aspects of attacks on DRL using realistic and efficient attacks. First, we adapt and propose efficient black-box attacks when we do not have access to DRL model parameters. Second, to address the high computational demands of existing attacks, we introduce efficient online sequential attacks that exploit temporal consistency across consecutive steps. Third, we explore the possibility of an attacker perturbing other aspects in the DRL setting, such as the environment dynamics. Finally, to account for imperfections in how an attacker would inject perturbations in the physical world, we devise a method for generating a robust physical perturbations to be printed. The attack is evaluated on a real-world robot under various conditions. We conduct extensive experiments both in simulation such as Atari games, robotics and autonomous driving, and on real-world robotics, to compare the effectiveness of the proposed attacks with baseline approaches. To the best of our knowledge, we are the first to apply adversarial attacks on DRL systems to physical robots.