# Beware Untrusted Simulators -- Reward-Free Backdoor Attacks in Reinforcement Learning

arXiv: https://arxiv.org/abs/2602.05089 · HTML: https://mlsanity.com/paper/2602.05089/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods.
- Builds on 19 of its 32 references that are in the map.
- First posted on arXiv 4 Feb 2026.

## Placed in

- Deep reinforcement learning methods / [Adversarial robustness and RL in finance](https://mlsanity.com/deeprl/adversarial-robustness-and-rl-in-finance/)

## Builds on (most cited first, 19 of 19)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [Learning Dexterous In-Hand Manipulation](https://mlsanity.com/paper/1808.00177/) (2018)
- [Action Robust Reinforcement Learning and Applications in Continuous Control](https://mlsanity.com/paper/1901.09184/) (2019)
- [Physically Realizable Adversarial Examples for LiDAR Object Detection](https://mlsanity.com/paper/2004.00543/) (2020)
- [Subpopulation Data Poisoning Attacks](https://mlsanity.com/paper/2006.14026/) (2020)
- [BACKDOORL: Backdoor Attack against Competitive Reinforcement Learning](https://mlsanity.com/paper/2105.00579/) (2021)
- [LeVERB: Humanoid Whole-Body Control with Latent Vision-Language Instruction](https://mlsanity.com/paper/2506.13751/) (2025)
- [A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Models](https://mlsanity.com/paper/2502.13187/) (2025)
- [TrojDRL: Trojan Attacks on Deep Reinforcement Learning Agents](https://mlsanity.com/paper/1903.06638/) (2019)
- [SleeperNets: Universal Backdoor Poisoning Attacks Against Reinforcement Learning Agents](https://mlsanity.com/paper/2405.20539/) (2024)
- [Provable Defense against Backdoor Policies in Reinforcement Learning](https://mlsanity.com/paper/2211.10530/) (2022)
- [Rethinking Out-of-Distribution Detection for Reinforcement Learning: Advancing Methods for Evaluation and Detection](https://mlsanity.com/paper/2404.07099/) (2024)
- [Adversarial Inception Backdoor Attacks against Reinforcement Learning](https://mlsanity.com/paper/2410.13995/) (2024)
- [UNIDOOR: A Universal Framework for Action-Level Backdoor Attacks in Deep Reinforcement Learning](https://mlsanity.com/paper/2501.15529/) (2025)
- [TrojanTO: Action-Level Backdoor Attacks against Trajectory Optimization Models](https://mlsanity.com/paper/2506.12815/) (2025)

## Built on by (most cited first, 2 of 2)

- [Targeting World Models to Compromise Robot Learning Pipelines](https://mlsanity.com/paper/2606.09499/) (2026)
- [When the World Lies: Backdoor Attacks on Latent World Models for Downstream Control](https://mlsanity.com/paper/2609.15781/) (2026)

## Abstract (arXiv)

Simulated environments are a key piece in the success of Reinforcement Learning (RL), allowing practitioners and researchers to train decision making agents without running expensive experiments on real hardware. Simulators remain a security blind spot, however, enabling adversarial developers to alter the dynamics of their released simulators for malicious purposes. Therefore, in this work we highlight a novel threat, demonstrating how simulator dynamics can be exploited to stealthily implant action-level backdoors into RL agents. The backdoor then allows an adversary to reliably activate targeted actions in an agent upon observing a predefined ``trigger'', leading to potentially dangerous consequences. Traditional backdoor attacks are limited in their strong threat models, assuming the adversary has near full control over an agent's training pipeline, enabling them to both alter and observe agent's rewards. As these assumptions are infeasible to implement within a simulator, we propose a new attack ``Daze'' which is able to reliably and stealthily implant backdoors into RL agents trained for real world tasks without altering or even observing their rewards. We provide formal proof of Daze's effectiveness in guaranteeing attack success across general RL tasks along with extensive empirical evaluations on both discrete and continuous action space domains. We additionally provide the first example of RL backdoor attacks transferring to real, robotic hardware. These developments motivate further research into securing all components of the RL training pipeline to prevent malicious attacks.