Provably Efficient Action-Manipulation Attack Against Continuous Reinforcement Learning
On the map: Deep reinforcement learning methods → Adversarial robustness and RL in finance
Abstract (arXiv)
Manipulating the interaction trajectories between the intelligent agent and the environment can control the agent's training and behavior, exposing the potential vulnerabilities of reinforcement learning (RL). For example, in Cyber-Physical Systems (CPS) controlled by RL, the attacker can manipulate the actions of the adopted RL to other actions during the training phase, which will lead to bad consequences. Existing work has studied action-manipulation attacks in tabular settings, where the states and actions are discrete. As seen in many up-and-coming RL applications, such as autonomous driving, continuous action space is widely accepted, however, its action-manipulation attacks have not been thoroughly investigated yet. In this paper, we consider this crucial problem in both white-box and black-box scenarios. Specifically, utilizing the knowledge derived exclusively from trajectories, we propose a black-box attack algorithm named LCBT, which uses the Monte Carlo tree search method for efficient action searching and manipulation. Additionally, we demonstrate that for an agent whose dynamic regret is sub-linearly related to the total number of steps, LCBT can teach the agent to converge to target policies with only sublinear attack cost, i.e., Oleft(R(T) + MH3KElog (MT)right)(0<E<1), where H is the number of steps per episode, K is the total number of episodes, T=KH is the total number of steps, M is the number of subspaces divided in the state space, and R(T) is the bound of the RL algorithm's regret. We conduct our proposed attack methods on three aggressive algorithms: DDPG, PPO, and TD3 in continuous settings, which show a promising attack performance.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods. Builds on 33 of its 38 references that are in the map. First posted on arXiv 20 Nov 2024.
Builds on
33 of 38 references.
References · 33
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2015Continuous control with deep reinforcement learningcited by 3,407
- 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
- 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 721
- 2017Adversarial Attacks on Neural Network Policiescited by 320
- 2019Reinforcement Learning in Healthcare: A Surveycited by 310
- 2021A Practical Guide to Multi-Objective Reinforcement Learning and Planningcited by 256
- 2019Action Robust Reinforcement Learning and Applications in Continuous Controlcited by 174
- 2017Tactics of Adversarial Attack on Deep Reinforcement Learning Agentscited by 166
- 2021Robust Reinforcement Learning on State Observations with Learned Optimal Adversarycited by 135
- 2017Vulnerability of Deep Reinforcement Learning to Policy Induction Attackscited by 91
- 2020Adaptive Reward-Poisoning Attacks against Reinforcement Learningcited by 86
21 more
- 2019Policy Poisoning in Batch Reinforcement Learning and Controlcited by 70
- 2017Continuous State-Space Models for Optimal Sepsis Treatment - a Deep Reinforcement Learning Approachcited by 68
- 2020Challenges and Countermeasures for Adversarial Attacks on Deep Reinforcement Learningcited by 61
- 2019Deceptive Reinforcement Learning Under Adversarial Manipulations on Cost Signalscited by 50
- 2020Vulnerability-Aware Poisoning Mechanism for Online RL with Unknown Dynamicscited by 38
- 2020Dynamic Regret of Policy Optimization in Non-stationary Environmentscited by 33
- 2019Characterizing Attacks on Deep Reinforcement Learningcited by 31
- 2020Robustifying Reinforcement Learning Agents via Action Space Adversarial Trainingcited by 30
- 2021Provably Efficient Black-Box Action Poisoning Attacks Against Reinforcement Learningcited by 29
- 2019Spatiotemporally Constrained Action Space Attacks on Deep Reinforcement Learning Agentscited by 25
- 2018Data Poisoning Attacks in Contextual Banditscited by 23
- 2018Financial Trading as a Game: A Deep Reinforcement Learning Approachcited by 22
- 2022Reward Poisoning Attacks on Offline Multi-Agent Reinforcement Learningcited by 20
- 2019Simple Physical Adversarial Examples against End-to-End Autonomous Driving Modelscited by 17
- 2020Query-based Targeted Action-Space Adversarial Policies on Deep Reinforcement Learning Agentscited by 15
- 2020Enhanced Adversarial Strategically-Timed Attacks against Deep Reinforcement Learningcited by 13
- 2023Optimal Attack and Defense for Reinforcement Learningcited by 12
- 2023Efficient Adversarial Attacks on Online Multi-agent Reinforcement Learningcited by 10
- 2020Inverse Filtering for Hidden Markov Models with Applications to Counter-Adversarial Autonomous Systemscited by 8
- 2022Execute Order 66: Targeted Data Poisoning for Reinforcement Learningcited by 7
- 2021Disturbing Reinforcement Learning Agents with Corrupted Rewardscited by 4
Built on by
Citing papers per year: 2025: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025TrojanTO: Action-Level Backdoor Attacks against Trajectory Optimization Models
- 2024Online Poisoning Attack Against Reinforcement Learning under Black-box Environments
- 2026Demonstration-Guided Observation Attacks on Black-Box Safe Reinforcement Learning Controllers for Robotic Systems
- 2025Exposing Vulnerabilities in RL: A Novel Stealthy Backdoor Attack through Reward Poisoning ↗
- 2022Security of Deep Reinforcement Learning for Autonomous Driving: A Survey
- 2022COPA: Certifying Robust Policies for Offline Reinforcement Learning against Poisoning Attacks
- 2026When to Intervene? State-Aware Sparse Manipulation in Federated Reinforcement Learning
- 2026Fundamental Limits of Man-in-the-Middle Attack Detection in Model-Free Reinforcement Learning