# TrojDRL: Trojan Attacks on Deep Reinforcement Learning Agents

arXiv: https://arxiv.org/abs/1903.06638 · HTML: https://mlsanity.com/paper/1903.06638/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Mar 2019.

## Built on by (most cited first, 30 of 30)

- [Topological Detection of Trojaned Neural Networks](https://mlsanity.com/paper/2106.06469/) (2021)
- [BadRL: Sparse Targeted Backdoor Attack Against Reinforcement Learning](https://mlsanity.com/paper/2312.12585/) (2023)
- [SleeperNets: Universal Backdoor Poisoning Attacks Against Reinforcement Learning Agents](https://mlsanity.com/paper/2405.20539/) (2024)
- [A Survey of Neural Trojan Attacks and Defenses in Deep Learning](https://mlsanity.com/paper/2202.07183/) (2022)
- [Attacking Cooperative Multi-Agent Reinforcement Learning by Adversarial Minority Influence](https://mlsanity.com/paper/2302.03322/) (2023)
- [Efficient Reward Poisoning Attacks on Online Deep Reinforcement Learning](https://mlsanity.com/paper/2205.14842/) (2022)
- [Adversarial Inception Backdoor Attacks against Reinforcement Learning](https://mlsanity.com/paper/2410.13995/) (2024)
- [Execute Order 66: Targeted Data Poisoning for Reinforcement Learning](https://mlsanity.com/paper/2201.00762/) (2022)
- [Neighboring Backdoor Attacks on Graph Convolutional Network](https://mlsanity.com/paper/2201.06202/) (2022)
- [Data Free Backdoor Attacks](https://mlsanity.com/paper/2412.06219/) (2024)
- [Local Environment Poisoning Attacks on Federated Reinforcement Learning](https://mlsanity.com/paper/2303.02725/) (2023)
- [Deep-Attack over the Deep Reinforcement Learning](https://mlsanity.com/paper/2205.00807/) (2022)
- [Trojan Horse Training for Breaking Defenses against Backdoor Attacks in Deep Learning](https://mlsanity.com/paper/2203.15506/) (2022)
- [Backdoor Attacks on Multiagent Collaborative Systems](https://mlsanity.com/paper/2211.11455/) (2022)
- [TrojanTO: Action-Level Backdoor Attacks against Trajectory Optimization Models](https://mlsanity.com/paper/2506.12815/) (2025)
- [Advancing Robustness in Deep Reinforcement Learning with an Ensemble Defense Approach](https://mlsanity.com/paper/2507.17070/) (2025)
- [TRAPDOOR: Repurposing backdoors to detect dataset bias in machine learning-based genomic analysis](https://mlsanity.com/paper/2108.10132/) (2021)
- [Backdoors in DRL: Four Environments Focusing on In-distribution Triggers](https://mlsanity.com/paper/2505.17248/) (2025)
- [Beware Untrusted Simulators -- Reward-Free Backdoor Attacks in Reinforcement Learning](https://mlsanity.com/paper/2602.05089/) (2026)
- [Investigating the Treacherous Turn in Deep Reinforcement Learning](https://mlsanity.com/paper/2504.08943/) (2025)
- [Targeting World Models to Compromise Robot Learning Pipelines](https://mlsanity.com/paper/2606.09499/) (2026)
- [Game of Trojans: A Submodular Byzantine Approach](https://mlsanity.com/paper/2207.05937/) (2022)
- [Recover Triggered States: Protect Model Against Backdoor Attack in Reinforcement Learning](https://mlsanity.com/paper/2304.00252/) (2023)
- [Trading Devil RL: Backdoor attack via Stock market, Bayesian Optimization and Reinforcement Learning](https://mlsanity.com/paper/2412.17908/) (2024)
- [Towards Robust Deep Reinforcement Learning against Environmental State Perturbation](https://mlsanity.com/paper/2506.08961/) (2025)
- [Optimal Perturbation Budget Allocation for Data Poisoning in Offline Reinforcement Learning](https://mlsanity.com/paper/2512.08485/) (2025)
- [When Backdoors Meet Partial Observability: Attacking Real-World Reinforcement Learning](https://mlsanity.com/paper/2601.14104/) (2026)
- [BehaviorGuard: Online Backdoor Defense for Deep Reinforcement Learning](https://mlsanity.com/paper/2605.05977/) (2026)
- [Test-time Adversarial Takeover: A Real-time Hijacking Interface against Robotic Diffusion Policies](https://mlsanity.com/paper/2606.10371/) (2026)
- [TrojanWorld: Backdooring World-Model Agents via Imagination Steering](https://mlsanity.com/paper/2609.07051/) (2026)

## Abstract (arXiv)

Recent work has identified that classification models implemented as neural networks are vulnerable to data-poisoning and Trojan attacks at training time. In this work, we show that these training-time vulnerabilities extend to deep reinforcement learning (DRL) agents and can be exploited by an adversary with access to the training process. In particular, we focus on Trojan attacks that augment the function of reinforcement learning policies with hidden behaviors. We demonstrate that such attacks can be implemented through minuscule data poisoning (as little as 0.025% of the training data) and in-band reward modification that does not affect the reward on normal inputs. The policies learned with our proposed attack approach perform imperceptibly similar to benign policies but deteriorate drastically when the Trojan is triggered in both targeted and untargeted settings. Furthermore, we show that existing Trojan defense mechanisms for classification tasks are not effective in the reinforcement learning setting.