Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learning
Not on the map. Why?
Abstract (arXiv)
Adversarial attacks against conventional Deep Learning (DL) systems and algorithms have been widely studied, and various defenses were proposed. However, the possibility and feasibility of such attacks against Deep Reinforcement Learning (DRL) are less explored. As DRL has achieved great success in various complex tasks, designing effective adversarial attacks is an indispensable prerequisite towards building robust DRL algorithms. In this paper, we introduce two novel adversarial attack techniques to stealthily and efficiently attack the DRL agents. These two techniques enable an adversary to inject adversarial samples in a minimal set of critical moments while causing the most severe damage to the agent. The first technique is the critical point attack: the adversary builds a model to predict the future environmental states and agent's actions, assesses the damage of each possible attack strategy, and selects the optimal one. The second technique is the antagonist attack: the adversary automatically learns a domain-agnostic model to discover the critical moments of attacking the agent in an episode. Experimental results demonstrate the effectiveness of our techniques. Specifically, to successfully attack the DRL agent, our critical point technique only requires 1 (TORCS) or 2 (Atari Pong and Breakout) steps, and the antagonist technique needs fewer than 5 steps (4 Mujoco tasks), which are significant improvements over state-of-the-art methods.
Key facts. Cited by 62 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 May 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 7; 2022: 13; 2023: 10; 2024: 10; 2025: 14; 2026: 8.
62 papers.
Sort:
By area:
- 2021Defending Against Backdoor Attacks in Natural Language Generationcited by 30
- 2023Adversarial Robust Deep Reinforcement Learning Requires Redefining Robustnesscited by 18
- 2024Diffusion Policy Attacker: Crafting Adversarial Attacks for Diffusion-based Policiescited by 17
- 2022RoMFAC: A robust mean-field actor-critic reinforcement learning against adversarial perturbations on statescited by 16
- 2022Robust Deep Reinforcement Learning through Bootstrapped Opportunistic Curriculumcited by 15
- 2023Implicit Poisoning Attacks in Two-Agent Reinforcement Learning: Adversarial Policies for Training-Time Attackscited by 14
- 2021Understanding Adversarial Attacks on Observations in Deep Reinforcement Learningcited by 13
- 2021Targeted Attack on Deep RL-based Autonomous Driving with Learned Visual Patternscited by 13
- 2022Illusory Attacks: Information-Theoretic Detectability Matters in Adversarial Attackscited by 12
- 2023Optimal Attack and Defense for Reinforcement Learningcited by 12
- 2021Deep Reinforcement Learning Policies Learn Shared Adversarial Features Across MDPscited by 11
- 2021Resilient Machine Learning for Networked Cyber Physical Systems: A Survey for Machine Learning Security to Securing Machine Learning for CPScited by 10
38 more
- 2025Rethinking Adversarial Attacks in Reinforcement Learning from Policy Distribution Perspectivecited by 10
- 2021Real-time Adversarial Perturbations against Deep Reinforcement Learning Policies: Attacks and Defensescited by 9
- 2022Sparse Adversarial Attack in Multi-agent Reinforcement Learningcited by 8
- 2022Defending Observation Attacks in Deep Reinforcement Learning via Detection and Denoisingcited by 8
- 2022New Challenges in Reinforcement Learning: A Survey of Security and Privacycited by 7
- 2023Rethinking Adversarial Policies: A Generalized Attack Formulation and Provable Defense in RLcited by 7
- 2024SUB-PLAY: Adversarial Policies against Partially Observed Multi-Agent Reinforcement Learning Systemscited by 7
- 2023Detecting Adversarial Directions in Deep Reinforcement Learning to Make Robust Decisionscited by 6
- 2023ReRoGCRL: Representation-based Robustness in Goal-Conditioned Reinforcement Learningcited by 6
- 2022Deep-Attack over the Deep Reinforcement Learningcited by 5
- 2025UNIDOOR: A Universal Framework for Action-Level Backdoor Attacks in Deep Reinforcement Learningcited by 5
- 2024Corruption-Robust Offline Two-Player Zero-Sum Markov Gamescited by 4
- 2026SoK: The Pitfalls of Deep Reinforcement Learning for Cybersecuritycited by 4
- 2021Online Adversarial Attackscited by 3
- 2023SoK: Adversarial Machine Learning Attacks and Defences in Multi-Agent Reinforcement Learningcited by 3
- 2023Regret-Based Defense in Adversarial Reinforcement Learningcited by 3
- 2023Toward Evaluating Robustness of Reinforcement Learning with Adversarial Policycited by 3
- 2025Advancing Robustness in Deep Reinforcement Learning with an Ensemble Defense Approachcited by 3
- 2022Imitating Opponent to Win: Adversarial Policy Imitation Learning in Two-player Competitive Gamescited by 2
- 2024Robust Deep Reinforcement Learning against Adversarial Behavior Manipulationcited by 2
- 2024On Minimizing Adversarial Counterfactual Error in Adversarial RLcited by 2
- 2025How Vulnerable Is My Learned Policy? Universal Adversarial Perturbation Attacks On Modern Behavior Cloning Policiescited by 2
- 2025Policy Teaching via Data Poisoning in Learning from Human Preferencescited by 2
- 2025Sharpening the Spear: Adaptive Expert-Guided Adversarial Attack Against DRL-based Autonomous Driving Policiescited by 2
- 2022Ad Hoc Teamwork in the Presence of Adversariescited by 1
- 2022A Transferable and Automatic Tuning of Deep Reinforcement Learning for Cost Effective Phishing Detectioncited by 1
- 2022Security of Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 1
- 2024CuDA2: An approach for Incorporating Traitor Agents into Cooperative Multi-Agent Systemscited by 1
- 2025CAMP in the Odyssey: Provably Robust Reinforcement Learning with Certified Radius Maximizationcited by 1
- 2025Backdoors in DRL: Four Environments Focusing on In-distribution Triggerscited by 1
- 2025Robust Driving Control for Autonomous Vehicles: An Intelligent General-sum Constrained Adversarial Reinforcement Learning Approachcited by 1
- 2026Angel or Demon: Investigating the Plasticity Interventions' Impact on Backdoor Threats in Deep Reinforcement Learningcited by 1
- 2022Explainable and Safe Reinforcement Learning for Autonomous Air Mobilitycited by 0
- 2022Certifying Safety in Reinforcement Learning under Adversarial Perturbation Attackscited by 0
- 2023PGN: A perturbation generation network against deep reinforcement learningcited by 0
- 2024Adaptive Discounting of Training Time Attackscited by 0
- 2024On the Perturbed States for Transformed Input-robust Reinforcement Learningcited by 0
- 2024Robust off-policy Reinforcement Learning via Soft Constrained Adversarycited by 0
50 most cited of 62 All 62 (JSON)
- 2026Gimitest: A Comprehensive Tool for Testing Reinforcement Learning Policies8 Jul 2026
- 2026Off the Rails: Hijacking the Scoring Head in Generative End-to-End Driving Planners with Safety-Violating Adversarial Perturbations29 Jun 2026
- 2026RoAd-RL: A Unified Library and Benchmark for Robust Adversarial Reinforcement Learning29 Jun 2026
- 2026Angel or Demon: Investigating the Plasticity Interventions' Impact on Backdoor Threats in Deep Reinforcement Learning14 May 2026
- 2026Finding the Weakest Link: Adversarial Attack against Multi-Agent Communications13 May 2026
- 2026Efficient Preference Poisoning Attack on Offline RLHF4 May 2026
- 2026Advantage-based Temporal Attack in Reinforcement Learning23 Feb 2026
- 2026SoK: The Pitfalls of Deep Reinforcement Learning for Cybersecurity9 Feb 2026
- 2025Sparse Offline Reinforcement Learning with Corruption Robustness31 Dec 2025
- 2025CFLight: Enhancing Safety with Traffic Signal Control through Counterfactual Learning10 Dec 2025
- 2025Robust Driving Control for Autonomous Vehicles: An Intelligent General-sum Constrained Adversarial Reinforcement Learning Approach10 Oct 2025
- 2025Advancing Robustness in Deep Reinforcement Learning with an Ensemble Defense Approach22 Jul 2025
38 more
- 2025Sharpening the Spear: Adaptive Expert-Guided Adversarial Attack Against DRL-based Autonomous Driving Policies23 Jun 2025
- 2025Off-Policy Actor-Critic for Adversarial Observation Robustness: Virtual Alternative Training via Symmetric Policy Evaluation20 Jun 2025
- 2025Collapsing Sequence-Level Data-Policy Coverage via Poisoning Attack in Offline Reinforcement Learning12 Jun 2025
- 2025Robust In-Context Reinforcement Learning Under Reward Poisoning Attacks7 Jun 2025
- 2025Backdoors in DRL: Four Environments Focusing on In-distribution Triggers22 May 2025
- 2025Policy Teaching via Data Poisoning in Learning from Human Preferences13 Mar 2025
- 2025How Vulnerable Is My Learned Policy? Universal Adversarial Perturbation Attacks On Modern Behavior Cloning Policies6 Feb 2025
- 2025CAMP in the Odyssey: Provably Robust Reinforcement Learning with Certified Radius Maximization29 Jan 2025
- 2025UNIDOOR: A Universal Framework for Action-Level Backdoor Attacks in Deep Reinforcement Learning26 Jan 2025
- 2025Rethinking Adversarial Attacks in Reinforcement Learning from Policy Distribution Perspective7 Jan 2025
- 2024Criticality and Safety Margins for Reinforcement Learning26 Sep 2024
- 2024Robust off-policy Reinforcement Learning via Soft Constrained Adversary31 Aug 2024
- 2024On the Perturbed States for Transformed Input-robust Reinforcement Learning31 Jul 2024
- 2024CuDA2: An approach for Incorporating Traitor Agents into Cooperative Multi-Agent Systems25 Jun 2024
- 2024On Minimizing Adversarial Counterfactual Error in Adversarial RL7 Jun 2024
- 2024Robust Deep Reinforcement Learning against Adversarial Behavior Manipulation6 Jun 2024
- 2024Diffusion Policy Attacker: Crafting Adversarial Attacks for Diffusion-based Policies29 May 2024
- 2024Corruption-Robust Offline Two-Player Zero-Sum Markov Games4 Mar 2024
- 2024SUB-PLAY: Adversarial Policies against Partially Observed Multi-Agent Reinforcement Learning Systems6 Feb 2024
- 2024Adaptive Discounting of Training Time Attacks5 Jan 2024
- 2023PGN: A perturbation generation network against deep reinforcement learning20 Dec 2023
- 2023ReRoGCRL: Representation-based Robustness in Goal-Conditioned Reinforcement Learning12 Dec 2023
- 2023Optimal Attack and Defense for Reinforcement Learning30 Nov 2023
- 2023Detecting Adversarial Directions in Deep Reinforcement Learning to Make Robust Decisions9 Jun 2023
- 2023Rethinking Adversarial Policies: A Generalized Attack Formulation and Provable Defense in RL27 May 2023
- 2023Toward Evaluating Robustness of Reinforcement Learning with Adversarial Policy4 May 2023
- 2023Implicit Poisoning Attacks in Two-Agent Reinforcement Learning: Adversarial Policies for Training-Time Attacks27 Feb 2023
- 2023Regret-Based Defense in Adversarial Reinforcement Learning14 Feb 2023
- 2023Adversarial Robust Deep Reinforcement Learning Requires Redefining Robustness17 Jan 2023
- 2023SoK: Adversarial Machine Learning Attacks and Defences in Multi-Agent Reinforcement Learning11 Jan 2023
- 2022New Challenges in Reinforcement Learning: A Survey of Security and Privacy31 Dec 2022
- 2022Certifying Safety in Reinforcement Learning under Adversarial Perturbation Attacks28 Dec 2022
- 2022Security of Deep Reinforcement Learning for Autonomous Driving: A Survey12 Dec 2022
- 2022Explainable and Safe Reinforcement Learning for Autonomous Air Mobility24 Nov 2022
- 2022Imitating Opponent to Win: Adversarial Policy Imitation Learning in Two-player Competitive Games30 Oct 2022
- 2022A Transferable and Automatic Tuning of Deep Reinforcement Learning for Cost Effective Phishing Detection19 Sep 2022
- 2022Ad Hoc Teamwork in the Presence of Adversaries9 Aug 2022
- 2022Illusory Attacks: Information-Theoretic Detectability Matters in Adversarial Attacks20 Jul 2022
50 newest of 62 All 62 (JSON)
- 2021Targeted Attack on Deep RL-based Autonomous Driving with Learned Visual Patternscited by 13
- 2025Sharpening the Spear: Adaptive Expert-Guided Adversarial Attack Against DRL-based Autonomous Driving Policiescited by 2
- 2022Security of Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 1
- 2025Robust Driving Control for Autonomous Vehicles: An Intelligent General-sum Constrained Adversarial Reinforcement Learning Approachcited by 1
- 2026Off the Rails: Hijacking the Scoring Head in Generative End-to-End Driving Planners with Safety-Violating Adversarial Perturbationscited by 0
- 2026Off the Rails: Hijacking the Scoring Head in Generative End-to-End Driving Planners with Safety-Violating Adversarial Perturbations29 Jun 2026
- 2025Robust Driving Control for Autonomous Vehicles: An Intelligent General-sum Constrained Adversarial Reinforcement Learning Approach10 Oct 2025
- 2025Sharpening the Spear: Adaptive Expert-Guided Adversarial Attack Against DRL-based Autonomous Driving Policies23 Jun 2025
- 2022Security of Deep Reinforcement Learning for Autonomous Driving: A Survey12 Dec 2022
- 2021Targeted Attack on Deep RL-based Autonomous Driving with Learned Visual Patterns16 Sep 2021
- 2022Imitating Opponent to Win: Adversarial Policy Imitation Learning in Two-player Competitive Gamescited by 2
- 2025How Vulnerable Is My Learned Policy? Universal Adversarial Perturbation Attacks On Modern Behavior Cloning Policiescited by 2
- 2025Collapsing Sequence-Level Data-Policy Coverage via Poisoning Attack in Offline Reinforcement Learningcited by 0
- 2025Sparse Offline Reinforcement Learning with Corruption Robustnesscited by 0
- 2025Sparse Offline Reinforcement Learning with Corruption Robustness31 Dec 2025
- 2025Collapsing Sequence-Level Data-Policy Coverage via Poisoning Attack in Offline Reinforcement Learning12 Jun 2025
- 2025How Vulnerable Is My Learned Policy? Universal Adversarial Perturbation Attacks On Modern Behavior Cloning Policies6 Feb 2025
- 2022Imitating Opponent to Win: Adversarial Policy Imitation Learning in Two-player Competitive Games30 Oct 2022
- 2021Defending Against Backdoor Attacks in Natural Language Generationcited by 30
- 2024Adaptive Discounting of Training Time Attackscited by 0
- 2024Adaptive Discounting of Training Time Attacks5 Jan 2024
- 2021Defending Against Backdoor Attacks in Natural Language Generation3 Jun 2021
- 2023Adversarial Robust Deep Reinforcement Learning Requires Redefining Robustnesscited by 18
- 2022RoMFAC: A robust mean-field actor-critic reinforcement learning against adversarial perturbations on statescited by 16
- 2022Robust Deep Reinforcement Learning through Bootstrapped Opportunistic Curriculumcited by 15
- 2023Implicit Poisoning Attacks in Two-Agent Reinforcement Learning: Adversarial Policies for Training-Time Attackscited by 14
- 2021Understanding Adversarial Attacks on Observations in Deep Reinforcement Learningcited by 13
- 2022Illusory Attacks: Information-Theoretic Detectability Matters in Adversarial Attackscited by 12
- 2023Optimal Attack and Defense for Reinforcement Learningcited by 12
- 2021Deep Reinforcement Learning Policies Learn Shared Adversarial Features Across MDPscited by 11
- 2021Resilient Machine Learning for Networked Cyber Physical Systems: A Survey for Machine Learning Security to Securing Machine Learning for CPScited by 10
- 2025Rethinking Adversarial Attacks in Reinforcement Learning from Policy Distribution Perspectivecited by 10
- 2021Real-time Adversarial Perturbations against Deep Reinforcement Learning Policies: Attacks and Defensescited by 9
- 2022Sparse Adversarial Attack in Multi-agent Reinforcement Learningcited by 8
12 most cited of 48 All 48 (JSON)
- 2026Gimitest: A Comprehensive Tool for Testing Reinforcement Learning Policies8 Jul 2026
- 2026RoAd-RL: A Unified Library and Benchmark for Robust Adversarial Reinforcement Learning29 Jun 2026
- 2026Angel or Demon: Investigating the Plasticity Interventions' Impact on Backdoor Threats in Deep Reinforcement Learning14 May 2026
- 2026Finding the Weakest Link: Adversarial Attack against Multi-Agent Communications13 May 2026
- 2026Advantage-based Temporal Attack in Reinforcement Learning23 Feb 2026
- 2026SoK: The Pitfalls of Deep Reinforcement Learning for Cybersecurity9 Feb 2026
- 2025CFLight: Enhancing Safety with Traffic Signal Control through Counterfactual Learning10 Dec 2025
- 2025Advancing Robustness in Deep Reinforcement Learning with an Ensemble Defense Approach22 Jul 2025
- 2025Off-Policy Actor-Critic for Adversarial Observation Robustness: Virtual Alternative Training via Symmetric Policy Evaluation20 Jun 2025
- 2025Robust In-Context Reinforcement Learning Under Reward Poisoning Attacks7 Jun 2025
- 2025Backdoors in DRL: Four Environments Focusing on In-distribution Triggers22 May 2025
- 2025CAMP in the Odyssey: Provably Robust Reinforcement Learning with Certified Radius Maximization29 Jan 2025
12 newest of 48 All 48 (JSON)