# Backdoors in DRL: Four Environments Focusing on In-distribution Triggers

arXiv: https://arxiv.org/abs/2505.17248 · HTML: https://mlsanity.com/paper/2505.17248/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods.
- Builds on 23 of its 39 references that are in the map.
- First posted on arXiv 22 May 2025.

## Placed in

- Deep reinforcement learning methods / [Adversarial robustness and RL in finance](https://mlsanity.com/deeprl/adversarial-robustness-and-rl-in-finance/)

## Builds on (most cited first, 23 of 23)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [Spectral Signatures in Backdoor Attacks](https://mlsanity.com/paper/1811.00636/) (2018)
- [Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Tasks](https://mlsanity.com/paper/2306.13831/) (2023)
- [Adversarial Attacks on Neural Network Policies](https://mlsanity.com/paper/1702.02284/) (2017)
- [Adversarial Policies: Attacking Deep Reinforcement Learning](https://mlsanity.com/paper/1905.10615/) (2019)
- [Robust Deep Reinforcement Learning with Adversarial Attacks](https://mlsanity.com/paper/1712.03632/) (2017)
- [Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmark](https://mlsanity.com/paper/2310.12567/) (2023)
- [BACKDOORL: Backdoor Attack against Competitive Reinforcement Learning](https://mlsanity.com/paper/2105.00579/) (2021)
- [Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learning](https://mlsanity.com/paper/2005.07099/) (2020)
- [Stop-and-Go: Exploring Backdoor Attacks on Deep Reinforcement Learning-based Traffic Congestion Control Systems](https://mlsanity.com/paper/2003.07859/) (2020)
- [TrojViT: Trojan Insertion in Vision Transformers](https://mlsanity.com/paper/2208.13049/) (2022)
- [TrojDRL: Trojan Attacks on Deep Reinforcement Learning Agents](https://mlsanity.com/paper/1903.06638/) (2019)
- [Poisoning Deep Reinforcement Learning Agents with In-Distribution Triggers](https://mlsanity.com/paper/2106.07798/) (2021)
- [PolicyCleanse: Backdoor Detection and Mitigation in Reinforcement Learning](https://mlsanity.com/paper/2202.03609/) (2022)
- [Provable Defense against Backdoor Policies in Reinforcement Learning](https://mlsanity.com/paper/2211.10530/) (2022)
- [Towards A Critical Evaluation of Robustness for Deep Learning Backdoor Countermeasures](https://mlsanity.com/paper/2204.06273/) (2022)
- [Efficient Reward Poisoning Attacks on Online Deep Reinforcement Learning](https://mlsanity.com/paper/2205.14842/) (2022)
- [A Synergetic Attack against Neural Network Classifiers combining Backdoor and Adversarial Examples](https://mlsanity.com/paper/2109.01275/) (2021)
- [Attention Hijacking in Trojan Transformers](https://mlsanity.com/paper/2208.04946/) (2022)
- [Don't Watch Me: A Spatio-Temporal Trojan Attack on Deep-Reinforcement-Learning-Augment Autonomous Driving](https://mlsanity.com/paper/2211.14440/) (2022)

## Built on by (most cited first, 2 of 2)

- [Safety in Embodied AI: A Survey of Risks, Attacks, and Defenses](https://mlsanity.com/paper/2605.02900/) (2026)
- [CS-GBA: A Critical Sample-based Gradient-guided Backdoor Attack for Offline Reinforcement Learning](https://mlsanity.com/paper/2601.10407/) (2026)

## Abstract (arXiv)

Backdoor attacks, or trojans, pose a security risk by concealing undesirable behavior in deep neural network models. Open-source neural networks are downloaded from the internet daily, possibly containing backdoors, and third-party model developers are common. To advance research on backdoor attack mitigation, we develop several trojans for deep reinforcement learning (DRL) agents. We focus on in-distribution triggers, which occur within the agent's natural data distribution, since they pose a more significant security threat than out-of-distribution triggers due to their ease of activation by the attacker during model deployment. We implement backdoor attacks in four reinforcement learning (RL) environments: LavaWorld, Randomized LavaWorld, Colorful Memory, and Modified Safety Gymnasium. We train various models, both clean and backdoored, to characterize these attacks. We find that in-distribution triggers can require additional effort to implement and be more challenging for models to learn, but are nevertheless viable threats in DRL even using basic data poisoning attacks.