mlsanityData to 7 Oct 2026

Poisoning Deep Reinforcement Learning Agents with In-Distribution Triggers

arXiv 2106.07798 ↗First posted 14 Jun 2021cs.LG cs.CRcited by 19

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

In this paper, we propose a new data poisoning attack and apply it to deep reinforcement learning agents. Our attack centers on what we call in-distribution triggers, which are triggers native to the data distributions the model will be trained on and deployed in. We outline a simple procedure for embedding these, and other, triggers in deep reinforcement learning agents following a multi-task learning paradigm, and demonstrate in three common reinforcement learning environments. We believe that this work has important implications for the security of deep learning models.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 17 of its 25 references that are in the map. First posted on arXiv 14 Jun 2021.

Builds on

17 of 25 references.

References · 17

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2016OpenAI Gymcited by 1,684
  3. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  4. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  5. 2018Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networkscited by 500
  6. 2018Spectral Signatures in Backdoor Attackscited by 457
  7. 2018Detecting Backdoor Attacks on Deep Neural Networks by Activation Clusteringcited by 429
  8. 2020Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networkscited by 341
  9. 2020Weight Poisoning Attacks on Pre-trained Modelscited by 318
  10. 2019A backdoor attack against LSTM-based text classification systemscited by 214
  11. 2020PettingZoo: Gym for Multi-Agent Reinforcement Learningcited by 209
  12. 2019Detecting AI Trojans Using Meta Neural Analysiscited by 192
5 more
  1. 2020Blind Backdoors in Deep Learning Modelscited by 174
  2. 2019NeuronInspect: Detecting Backdoors in Neural Networks via Output Explanationscited by 49
  3. 2020The TrojAI Software Framework: An OpenSource tool for Embedding Trojans into Deep Learning Modelscited by 41
  4. 2020Natural Backdoor Attack on Text Datacited by 18
  5. 2019Invisible Backdoor Attacks on Deep Neural Networks via Steganography and Regularizationcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 1; 2024: 4; 2025: 7; 2026: 1.

19 papers.

Sort:

By area:

  1. 2023Poisoning Web-Scale Training Datasets is Practicalcited by 276
  2. 2022Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoningcited by 93
  3. 2022BAFFLE: Hiding Backdoors in Offline Reinforcement Learning Datasetscited by 24
  4. 2022Adversarial Cheap Talkcited by 16
  5. 2022A Temporal-Pattern Backdoor Attack to Deep Reinforcement Learningcited by 15
  6. 2025BLAST: A Stealthy Backdoor Leverage Attack against Cooperative Multi-Agent Deep Reinforcement Learning based Systemscited by 7
  7. 2024A Spatiotemporal Stealthy Backdoor Attack against Cooperative Multi-Agent Deep Reinforcement Learningcited by 6
  8. 2025Fox in the Henhouse: Supply-Chain Backdoor Attacks Against Reinforcement Learningcited by 5
  9. 2024Mitigating Deep Reinforcement Learning Backdoors in the Neural Activation Spacecited by 4
  10. 2022Don't Watch Me: A Spatio-Temporal Trojan Attack on Deep-Reinforcement-Learning-Augment Autonomous Drivingcited by 3
  11. 2024The last Dance : Robust backdoor attack via diffusion models and bayesian approachcited by 3
  12. 2024Online Poisoning Attack Against Reinforcement Learning under Black-box Environmentscited by 2
7 more
  1. 2025TooBadRL: Trigger Optimization to Boost Effectiveness of Backdoor Attacks on Deep Reinforcement Learningcited by 2
  2. 2025TrojanTO: Action-Level Backdoor Attacks against Trajectory Optimization Modelscited by 2
  3. 2022Security of Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 1
  4. 2025Investigating the Treacherous Turn in Deep Reinforcement Learningcited by 1
  5. 2025Backdoors in DRL: Four Environments Focusing on In-distribution Triggerscited by 1
  6. 2025Beyond Training-time Poisoning: Component-level and Post-training Backdoors in Deep Reinforcement Learningcited by 1
  7. 2026Auditing Near-Optimal Policies Can Be Exponentially Hard: Conditional Query Lower Bounds via Occupancy Rashomon Capacitycited by 0