mlsanityData to 14 Oct 2026

Execute Order 66: Targeted Data Poisoning for Reinforcement Learning

arXiv 2201.00762 ↗First posted 3 Jan 2022cs.LG cs.AI cs.CRcited by 7

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

Data poisoning for reinforcement learning has historically focused on general performance degradation, and targeted attacks have been successful via perturbations that involve control of the victim's policy and rewards. We introduce an insidious poisoning attack for reinforcement learning which causes agent misbehavior only at specific target states - all while minimally modifying a small fraction of training observations without assuming any control over policy or reward. We accomplish this by adapting a recent technique, gradient alignment, to reinforcement learning. We test our method and demonstrate success in two Atari games of varying difficulty.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 11 of its 20 references that are in the map. First posted on arXiv 3 Jan 2022.

Builds on

11 of 20 references.

References · 11

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  3. 2020Dataset Condensation with Gradient Matchingcited by 516
  4. 2017Towards Poisoning of Deep Learning Algorithms with Back-gradient Optimizationcited by 197
  5. 2021Unlearnable Examples: Making Personal Data Unexploitablecited by 193
  6. 2020Witches' Brew: Industrial Scale Data Poisoning via Gradient Matchingcited by 163
  7. 2019Deep Reinforcement Learning for Cyber Securitycited by 95
  8. 2019Learning to Confuse: Generating Training Time Adversarial Data with Auto-Encodercited by 70
  9. 2020Vulnerability-Aware Poisoning Mechanism for Online RL with Unknown Dynamicscited by 38
  10. 2021Preventing Unauthorized Use of Proprietary Data: Poisoning for Secure Dataset Releasecited by 35
  11. 2019TrojDRL: Trojan Attacks on Deep Reinforcement Learning Agentscited by 30

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 0; 2024: 5.

7 papers.

  1. 2022X-Risk Analysis for AI Researchcited by 60
  2. 2024TrajDeleter: Enabling Trajectory Forgetting in Offline Reinforcement Learning Agentscited by 7
  3. 2024Mitigating Deep Reinforcement Learning Backdoors in the Neural Activation Spacecited by 4
  4. 2022Security of Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 2
  5. 2024Online Poisoning Attack Against Reinforcement Learning under Black-box Environmentscited by 2
  6. 2024Provably Efficient Action-Manipulation Attack Against Continuous Reinforcement Learningcited by 1
  7. 2024Trading Devil RL: Backdoor attack via Stock market, Bayesian Optimization and Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2019Policy Poisoning in Batch Reinforcement Learning and Control
  2. 2025Optimal Perturbation Budget Allocation for Data Poisoning in Offline Reinforcement Learning
  3. 2023Data Poisoning to Fake a Nash Equilibrium in Markov Games
  4. 2025Policy Teaching via Data Poisoning in Learning from Human Preferences
  5. 2025Collapsing Sequence-Level Data-Policy Coverage via Poisoning Attack in Offline Reinforcement Learning
  6. 2026When Can You Poison Rewards? A Tight Characterization of Reward Poisoning in Linear MDPs
  7. 2021Poisoning Deep Reinforcement Learning Agents with In-Distribution Triggers
  8. 2024Is poisoning a real threat to LLM alignment? Maybe more so than you think