mlsanityData to 14 Oct 2026

BACKDOORL: Backdoor Attack against Competitive Reinforcement Learning

arXiv 2105.00579 ↗First posted 2 May 2021cs.CR cs.LGcited by 68

Not on the map. Why?

Cited by 20 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent research has confirmed the feasibility of backdoor attacks in deep reinforcement learning (RL) systems. However, the existing attacks require the ability to arbitrarily modify an agent's observation, constraining the application scope to simple RL systems such as Atari games. In this paper, we migrate backdoor attacks to more complex RL systems involving multiple agents and explore the possibility of triggering the backdoor without directly manipulating the agent's observation. As a proof of concept, we demonstrate that an adversary agent can trigger the backdoor of the victim agent with its own action in two-player competitive RL systems. We prototype and evaluate BACKDOORL in four competitive environments. The results show that when the backdoor is activated, the winning rate of the victim drops by 17% to 37% compared to when not activated.

Key facts. Cited by 68 papers in the map (data through 14 Oct 2026). Builds on 21 of its 46 references that are in the map. First posted on arXiv 2 May 2021.

Builds on

21 of 46 references.

References · 21

  1. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  2. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,106
  3. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,055
  4. 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 580
  5. 2018Detecting Backdoor Attacks on Deep Neural Networks by Activation Clusteringcited by 456
  6. 2019Hidden Trigger Backdoor Attackscited by 371
  7. 2018Analyzing Federated Learning through an Adversarial Lenscited by 351
  8. 2017Adversarial Attacks on Neural Network Policiescited by 320
  9. 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 311
  10. 2020Attack of the Tails: Yes, You Really Can Backdoor Federated Learningcited by 309
  11. 2019Label-Consistent Backdoor Attackscited by 296
  12. 2019Can You Really Backdoor Federated Learning?cited by 255
9 more
  1. 2019Adversarial Policies: Attacking Deep Reinforcement Learningcited by 214
  2. 2017Tactics of Adversarial Attack on Deep Reinforcement Learning Agentscited by 166
  3. 2019TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systemscited by 122
  4. 2017Emergent Complexity via Multi-Agent Competitioncited by 120
  5. 2019Gotta Catch 'Em All: Using Honeypots to Catch Adversarial Attacks on Neural Networkscited by 40
  6. 2020Stop-and-Go: Exploring Backdoor Attacks on Deep Reinforcement Learning-based Traffic Congestion Control Systemscited by 37
  7. 2020BAAAN: Backdoor Attacks Against Autoencoder and GAN-Based Machine Learning Modelscited by 25
  8. 2019Design of intentional backdoors in sequential modelscited by 21
  9. 2020Don't Trigger Me! A Triggerless Backdoor Attack Against Deep Neural Networkscited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 17; 2023: 10; 2024: 16; 2025: 13; 2026: 12.

68 papers.

Sort:

By area:

  1. 2024Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agentscited by 137
  2. 2023Personalization as a Shortcut for Few-Shot Backdoor Attack against Text-to-Image Diffusion Modelscited by 42
  3. 2024Test-Time Backdoor Attacks on Multimodal Large Language Modelscited by 40
  4. 2022Rethinking the Reverse-engineering of Trojan Triggerscited by 39
  5. 2022Training with More Confidence: Mitigating Injected and Natural Backdoors During Trainingcited by 38
  6. 2022Few-Shot Backdoor Attacks on Visual Object Trackingcited by 35
  7. 2024BadMerging: Backdoor Attacks Against Model Mergingcited by 34
  8. 2022Physical Backdoor Attacks to Lane Detection Systems in Autonomous Drivingcited by 31
  9. 2023Elijah: Eliminating Backdoors Injected in Diffusion Models via Distribution Shiftcited by 31
  10. 2023BadRL: Sparse Targeted Backdoor Attack Against Reinforcement Learningcited by 30
  11. 2022Imperceptible Backdoor Attack: From Input Space to Feature Representationcited by 27
  12. 2022COPA: Certifying Robust Policies for Offline Reinforcement Learning against Poisoning Attackscited by 26
38 more
  1. 2022BAFFLE: Hiding Backdoors in Offline Reinforcement Learning Datasetscited by 25
  2. 2022PolicyCleanse: Backdoor Detection and Mitigation in Reinforcement Learningcited by 19
  3. 2024SleeperNets: Universal Backdoor Poisoning Attacks Against Reinforcement Learning Agentscited by 19
  4. 2022Provable Defense against Backdoor Policies in Reinforcement Learningcited by 18
  5. 2023Attacking Cooperative Multi-Agent Reinforcement Learning by Adversarial Minority Influencecited by 17
  6. 2022A Temporal-Pattern Backdoor Attack to Deep Reinforcement Learningcited by 16
  7. 2022Adversarial Cheap Talkcited by 16
  8. 2023Implicit Poisoning Attacks in Two-Agent Reinforcement Learning: Adversarial Policies for Training-Time Attackscited by 14
  9. 2025Seven Security Challenges in Cross-domain Multi-agent LLM Systemscited by 12
  10. 2023ORL-AUDITOR: Dataset Auditing in Offline Deep Reinforcement Learningcited by 11
  11. 2023Efficient Adversarial Attacks on Online Multi-agent Reinforcement Learningcited by 10
  12. 2026State Backdoor: Towards Stealthy Real-world Poisoning Attack on Vision-Language-Action Model in State Spacecited by 9
  13. 2024Adversarial Inception Backdoor Attacks against Reinforcement Learningcited by 8
  14. 2024SUB-PLAY: Adversarial Policies against Partially Observed Multi-Agent Reinforcement Learning Systemscited by 7
  15. 2024Data Free Backdoor Attackscited by 7
  16. 2025BLAST: A Stealthy Backdoor Leverage Attack against Cooperative Multi-Agent Deep Reinforcement Learning based Systemscited by 7
  17. 2024Cooperative Backdoor Attack in Decentralized Reinforcement Learning with Theoretical Guaranteecited by 6
  18. 2024A Spatiotemporal Stealthy Backdoor Attack against Cooperative Multi-Agent Deep Reinforcement Learningcited by 6
  19. 2023Multi-Agent Reinforcement Learning: Methods, Applications, Visionary Prospects, and Challengescited by 5
  20. 2024TrojFM: Resource-efficient Backdoor Attacks against Very Large Foundation Modelscited by 5
  21. 2025UNIDOOR: A Universal Framework for Action-Level Backdoor Attacks in Deep Reinforcement Learningcited by 5
  22. 2025Fox in the Henhouse: Supply-Chain Backdoor Attacks Against Reinforcement Learningcited by 5
  23. 2022DECK: Model Hardening for Defending Pervasive Backdoorscited by 4
  24. 2022Invisible Backdoor Attack with Dynamic Triggers against Person Re-identificationcited by 4
  25. 2024Towards Optimal Adversarial Robust Q-learning with Bellman Infinity-errorcited by 4
  26. 2024Mitigating Deep Reinforcement Learning Backdoors in the Neural Activation Spacecited by 4
  27. 2025Temporal Logic-Based Multi-Vehicle Backdoor Attacks against Offline RL Agents in End-to-end Autonomous Drivingcited by 4
  28. 2022Emerging Threats in Deep Learning-Based Autonomous Driving: A Comprehensive Surveycited by 3
  29. 2022Backdoor Attacks on Multiagent Collaborative Systemscited by 3
  30. 2022Don't Watch Me: A Spatio-Temporal Trojan Attack on Deep-Reinforcement-Learning-Augment Autonomous Drivingcited by 3
  31. 2024Guarding the Gate: ConceptGuard Battles Concept-Level Backdoors in Concept Bottleneck Modelscited by 3
  32. 2025TooBadRL: Trigger Optimization to Boost Effectiveness of Backdoor Attacks on Deep Reinforcement Learningcited by 3
  33. 2025TrojanTO: Action-Level Backdoor Attacks against Trajectory Optimization Modelscited by 3
  34. 2022Security of Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 2
  35. 2023A Reminder of its Brittleness: Language Reward Shaping May Hinder Learning for Instruction Following Agentscited by 2
  36. 2025Backdoors in DRL: Four Environments Focusing on In-distribution Triggerscited by 2
  37. 2025Bridging Distribution Shift and AI Safety: Conceptual and Methodological Synergiescited by 2
  38. 2025PNAct: Crafting Backdoor Attacks in Safe Reinforcement Learningcited by 2

50 most cited of 68 All 68 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Exposing Vulnerabilities in RL: A Novel Stealthy Backdoor Attack through Reward Poisoning ↗
  2. 2023Recover Triggered States: Protect Model Against Backdoor Attack in Reinforcement Learning
  3. 2026When Backdoors Meet Partial Observability: Attacking Real-World Reinforcement Learning
  4. 2026Backdoors in RLVR: Jailbreak Backdoors in LLMs From Verifiable Reward
  5. 2022Backdoors Stuck At The Frontdoor: Multi-Agent Backdoor Attacks That Backfire
  6. 2026UBA-ORL: Unlearning-Activated Backdoor Attacks on Offline Reinforcement Learning ↗
  7. 2022Imitating Opponent to Win: Adversarial Policy Imitation Learning in Two-player Competitive Games
  8. 2025From Poisoned to Aware: Fostering Backdoor Self-Awareness in LLMs