# BadRL: Sparse Targeted Backdoor Attack Against Reinforcement Learning

arXiv: https://arxiv.org/abs/2312.12585 · HTML: https://mlsanity.com/paper/2312.12585/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 19 references that are in the map.
- First posted on arXiv 19 Dec 2023.

## Builds on (most cited first, 12 of 12)

- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [Hidden Trigger Backdoor Attacks](https://mlsanity.com/paper/1910.00033/) (2019)
- [Adversarial Policies: Attacking Deep Reinforcement Learning](https://mlsanity.com/paper/1905.10615/) (2019)
- [Just How Toxic is Data Poisoning? A Unified Benchmark for Backdoor and Data Poisoning Attacks](https://mlsanity.com/paper/2006.12557/) (2020)
- [Adaptive Reward-Poisoning Attacks against Reinforcement Learning](https://mlsanity.com/paper/2003.12613/) (2020)
- [Policy Teaching via Environment Poisoning: Training-time Adversarial Attacks against Reinforcement Learning](https://mlsanity.com/paper/2003.12909/) (2020)
- [BACKDOORL: Backdoor Attack against Competitive Reinforcement Learning](https://mlsanity.com/paper/2105.00579/) (2021)
- [Certified Robustness of Nearest Neighbors against Data Poisoning and Backdoor Attacks](https://mlsanity.com/paper/2012.03765/) (2020)
- [Vulnerability-Aware Poisoning Mechanism for Online RL with Unknown Dynamics](https://mlsanity.com/paper/2009.00774/) (2020)
- [TrojDRL: Trojan Attacks on Deep Reinforcement Learning Agents](https://mlsanity.com/paper/1903.06638/) (2019)
- [Provably Efficient Black-Box Action Poisoning Attacks Against Reinforcement Learning](https://mlsanity.com/paper/2110.04471/) (2021)
- [Reward Poisoning in Reinforcement Learning: Attacks Against Unknown Learners in Unknown Environments](https://mlsanity.com/paper/2102.08492/) (2021)

## Built on by (most cited first, 30 of 30)

- [Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agents](https://mlsanity.com/paper/2402.11208/) (2024)
- [SleeperNets: Universal Backdoor Poisoning Attacks Against Reinforcement Learning Agents](https://mlsanity.com/paper/2405.20539/) (2024)
- [BadMoE: Backdooring Mixture-of-Experts LLMs via Optimizing Routing Triggers and Infecting Dormant Experts](https://mlsanity.com/paper/2504.18598/) (2025)
- [State Backdoor: Towards Stealthy Real-world Poisoning Attack on Vision-Language-Action Model in State Space](https://mlsanity.com/paper/2601.04266/) (2026)
- [Adversarial Inception Backdoor Attacks against Reinforcement Learning](https://mlsanity.com/paper/2410.13995/) (2024)
- [BLAST: A Stealthy Backdoor Leverage Attack against Cooperative Multi-Agent Deep Reinforcement Learning based Systems](https://mlsanity.com/paper/2501.01593/) (2025)
- [Persistent Backdoor Attacks under Continual Fine-Tuning of LLMs](https://mlsanity.com/paper/2512.14741/) (2025)
- [Cooperative Backdoor Attack in Decentralized Reinforcement Learning with Theoretical Guarantee](https://mlsanity.com/paper/2405.15245/) (2024)
- [Recent Advances in Attack and Defense Approaches of Large Language Models](https://mlsanity.com/paper/2409.03274/) (2024)
- [UNIDOOR: A Universal Framework for Action-Level Backdoor Attacks in Deep Reinforcement Learning](https://mlsanity.com/paper/2501.15529/) (2025)
- [Temporal Logic-Based Multi-Vehicle Backdoor Attacks against Offline RL Agents in End-to-end Autonomous Driving](https://mlsanity.com/paper/2509.16950/) (2025)
- [TRAP: Tail-aware Ranking Attack for World-Model Planning](https://mlsanity.com/paper/2605.01950/) (2026)
- [TooBadRL: Trigger Optimization to Boost Effectiveness of Backdoor Attacks on Deep Reinforcement Learning](https://mlsanity.com/paper/2506.09562/) (2025)
- [TrojanTO: Action-Level Backdoor Attacks against Trajectory Optimization Models](https://mlsanity.com/paper/2506.12815/) (2025)
- [Collaborative Shadows: Distributed Backdoor Attacks in LLM-Based Multi-Agent Systems](https://mlsanity.com/paper/2510.11246/) (2025)
- [Robust Deep Reinforcement Learning against Adversarial Behavior Manipulation](https://mlsanity.com/paper/2406.03862/) (2024)
- [Online Poisoning Attack Against Reinforcement Learning under Black-box Environments](https://mlsanity.com/paper/2412.00797/) (2024)
- [PNAct: Crafting Backdoor Attacks in Safe Reinforcement Learning](https://mlsanity.com/paper/2507.00485/) (2025)
- [Which Experiences Are Influential for RL Agents? Efficiently Estimating The Influence of Experiences](https://mlsanity.com/paper/2405.14629/) (2024)
- [Beyond Training-time Poisoning: Component-level and Post-training Backdoors in Deep Reinforcement Learning](https://mlsanity.com/paper/2507.04883/) (2025)
- [Angel or Demon: Investigating the Plasticity Interventions' Impact on Backdoor Threats in Deep Reinforcement Learning](https://mlsanity.com/paper/2605.14587/) (2026)
- [A Disguised Wolf Is More Harmful Than a Toothless Tiger: Adaptive Malicious Code Injection Backdoor Attack Leveraging User Behavior as Triggers](https://mlsanity.com/paper/2408.10334/) (2024)
- [Trading Devil RL: Backdoor attack via Stock market, Bayesian Optimization and Reinforcement Learning](https://mlsanity.com/paper/2412.17908/) (2024)
- [CS-GBA: A Critical Sample-based Gradient-guided Backdoor Attack for Offline Reinforcement Learning](https://mlsanity.com/paper/2601.10407/) (2026)
- [When Backdoors Meet Partial Observability: Attacking Real-World Reinforcement Learning](https://mlsanity.com/paper/2601.14104/) (2026)
- [BehaviorGuard: Online Backdoor Defense for Deep Reinforcement Learning](https://mlsanity.com/paper/2605.05977/) (2026)
- [Plan2Cleanse: Test-Time Backdoor Defense via Monte-Carlo Planning in Deep Reinforcement Learning](https://mlsanity.com/paper/2605.09638/) (2026)
- [Target-Aware Interaction-Guided Reinforcement Learning for Black-Box Node Injection Attacks on Graph Neural Networks](https://mlsanity.com/paper/2607.04091/) (2026)
- [ORAN-DEFEND: Subspace Detection and Sanitization of Backdoor DRL xApps in Open RAN](https://mlsanity.com/paper/2607.06647/) (2026)
- [Securing Autonomous Vehicle Systems via Twin-Aware Federated Reinforcement Learning](https://mlsanity.com/paper/2607.08137/) (2026)

## Abstract (arXiv)

Backdoor attacks in reinforcement learning (RL) have previously employed intense attack strategies to ensure attack success. However, these methods suffer from high attack costs and increased detectability. In this work, we propose a novel approach, BadRL, which focuses on conducting highly sparse backdoor poisoning efforts during training and testing while maintaining successful attacks. Our algorithm, BadRL, strategically chooses state observations with high attack values to inject triggers during training and testing, thereby reducing the chances of detection. In contrast to the previous methods that utilize sample-agnostic trigger patterns, BadRL dynamically generates distinct trigger patterns based on targeted state observations, thereby enhancing its effectiveness. Theoretical analysis shows that the targeted backdoor attack is always viable and remains stealthy under specific assumptions. Empirical results on various classic RL tasks illustrate that BadRL can substantially degrade the performance of a victim agent with minimal poisoning efforts 0.003% of total training steps) during training and infrequent attacks during testing.