# A Temporal-Pattern Backdoor Attack to Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2205.02589 · HTML: https://mlsanity.com/paper/2205.02589/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 25 references that are in the map.
- First posted on arXiv 5 May 2022.

## Builds on (most cited first, 14 of 14)

- [Spectral Signatures in Backdoor Attacks](https://mlsanity.com/paper/1811.00636/) (2018)
- [Deep Recurrent Q-Learning for Partially Observable MDPs](https://mlsanity.com/paper/1507.06527/) (2015)
- [STRIP: A Defence Against Trojan Attacks on Deep Neural Networks](https://mlsanity.com/paper/1902.06531/) (2019)
- [Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering](https://mlsanity.com/paper/1811.03728/) (2018)
- [Backdoor Learning: A Survey](https://mlsanity.com/paper/2007.08745/) (2020)
- [Applications of Deep Reinforcement Learning in Communications and Networking: A Survey](https://mlsanity.com/paper/1810.07862/) (2018)
- [Deep Variational Reinforcement Learning for POMDPs](https://mlsanity.com/paper/1806.02426/) (2018)
- [Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review](https://mlsanity.com/paper/2007.10760/) (2020)
- [BACKDOORL: Backdoor Attack against Competitive Reinforcement Learning](https://mlsanity.com/paper/2105.00579/) (2021)
- [Do RNN and LSTM have Long Memory?](https://mlsanity.com/paper/2006.03860/) (2020)
- [Stop-and-Go: Exploring Backdoor Attacks on Deep Reinforcement Learning-based Traffic Congestion Control Systems](https://mlsanity.com/paper/2003.07859/) (2020)
- [Design of intentional backdoors in sequential models](https://mlsanity.com/paper/1902.09972/) (2019)
- [Poisoning Deep Reinforcement Learning Agents with In-Distribution Triggers](https://mlsanity.com/paper/2106.07798/) (2021)
- [POMDPs Make Better Hackers: Accounting for Uncertainty in Penetration Testing](https://mlsanity.com/paper/1307.8182/) (2013)

## Built on by (most cited first, 16 of 16)

- [Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agents](https://mlsanity.com/paper/2402.11208/) (2024)
- [BAFFLE: Hiding Backdoors in Offline Reinforcement Learning Datasets](https://mlsanity.com/paper/2210.04688/) (2022)
- [SleeperNets: Universal Backdoor Poisoning Attacks Against Reinforcement Learning Agents](https://mlsanity.com/paper/2405.20539/) (2024)
- [Adversarial Inception Backdoor Attacks against Reinforcement Learning](https://mlsanity.com/paper/2410.13995/) (2024)
- [BLAST: A Stealthy Backdoor Leverage Attack against Cooperative Multi-Agent Deep Reinforcement Learning based Systems](https://mlsanity.com/paper/2501.01593/) (2025)
- [Fox in the Henhouse: Supply-Chain Backdoor Attacks Against Reinforcement Learning](https://mlsanity.com/paper/2505.19532/) (2025)
- [Mitigating Deep Reinforcement Learning Backdoors in the Neural Activation Space](https://mlsanity.com/paper/2407.15168/) (2024)
- [Backdoor Attacks on Multiagent Collaborative Systems](https://mlsanity.com/paper/2211.11455/) (2022)
- [Don't Watch Me: A Spatio-Temporal Trojan Attack on Deep-Reinforcement-Learning-Augment Autonomous Driving](https://mlsanity.com/paper/2211.14440/) (2022)
- [TooBadRL: Trigger Optimization to Boost Effectiveness of Backdoor Attacks on Deep Reinforcement Learning](https://mlsanity.com/paper/2506.09562/) (2025)
- [Manipulating Trajectory Prediction with Backdoors](https://mlsanity.com/paper/2312.13863/) (2023)
- [Pruning Cannot Hurt Robustness: Certified Trade-offs in Reinforcement Learning](https://mlsanity.com/paper/2510.12939/) (2025)
- [CS-GBA: A Critical Sample-based Gradient-guided Backdoor Attack for Offline Reinforcement Learning](https://mlsanity.com/paper/2601.10407/) (2026)
- [BehaviorGuard: Online Backdoor Defense for Deep Reinforcement Learning](https://mlsanity.com/paper/2605.05977/) (2026)
- [Plan2Cleanse: Test-Time Backdoor Defense via Monte-Carlo Planning in Deep Reinforcement Learning](https://mlsanity.com/paper/2605.09638/) (2026)
- [Fast and Lightweight Backdoor Detection via Head Random Probing](https://mlsanity.com/paper/2605.18908/) (2026)

## Abstract (arXiv)

Deep reinforcement learning (DRL) has made significant achievements in many real-world applications. But these real-world applications typically can only provide partial observations for making decisions due to occlusions and noisy sensors. However, partial state observability can be used to hide malicious behaviors for backdoors. In this paper, we explore the sequential nature of DRL and propose a novel temporal-pattern backdoor attack to DRL, whose trigger is a set of temporal constraints on a sequence of observations rather than a single observation, and effect can be kept in a controllable duration rather than in the instant. We validate our proposed backdoor attack to a typical job scheduling task in cloud computing. Numerous experimental results show that our backdoor can achieve excellent effectiveness, stealthiness, and sustainability. Our backdoor's average clean data accuracy and attack success rate can reach 97.8% and 97.5%, respectively.