mlsanityData to 7 Oct 2026

UNIDOOR: A Universal Framework for Action-Level Backdoor Attacks in Deep Reinforcement Learning

arXiv 2501.15529 ↗First posted 26 Jan 2025cs.LG cs.AI cs.CRcited by 5

Not on the map. Why?

Abstract (arXiv)

Deep reinforcement learning (DRL) is widely applied to safety-critical decision-making scenarios. However, DRL is vulnerable to backdoor attacks, especially action-level backdoors, which pose significant threats through precise manipulation and flexible activation, risking outcomes like vehicle collisions or drone crashes. The key distinction of action-level backdoors lies in the utilization of the backdoor reward function to associate triggers with target actions. Nevertheless, existing studies typically rely on backdoor reward functions with fixed values or conditional flipping, which lack universality across diverse DRL tasks and backdoor designs, resulting in fluctuations or even failure in practice. This paper proposes the first universal action-level backdoor attack framework, called UNIDOOR, which enables adaptive exploration of backdoor reward functions through performance monitoring, eliminating the reliance on expert knowledge and grid search. We highlight that action tampering serves as a crucial component of action-level backdoor attacks in continuous action scenarios, as it addresses attack failures caused by low-frequency target actions. Extensive evaluations demonstrate that UNIDOOR significantly enhances the attack performance of action-level backdoors, showcasing its universality across diverse attack scenarios, including single/multiple agents, single/multiple backdoors, discrete/continuous action spaces, and sparse/dense reward signals. Furthermore, visualization results encompassing state distribution, neuron activation, and animations demonstrate the stealthiness of UNIDOOR. The source code of UNIDOOR can be found at https://github.com/maoubo/UNIDOOR.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 34 of its 60 references that are in the map. First posted on arXiv 26 Jan 2025.

Builds on

34 of 60 references.

References · 34

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  4. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  5. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  6. 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 550
  7. 2017Adversarial Attacks on Neural Network Policiescited by 314
  8. 2017Distral: Robust Multitask Reinforcement Learningcited by 231
  9. 2019Adversarial Policies: Attacking Deep Reinforcement Learningcited by 209
  10. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 158
  11. 2021Poisoning and Backdooring Contrastive Learningcited by 149
  12. 2019Demon in the Variant: Statistical Analysis of DNNs for Robust Backdoor Contamination Detectioncited by 121
22 more
  1. 2017Vulnerability of Deep Reinforcement Learning to Policy Induction Attackscited by 91
  2. 2021Backdoor Pre-trained Models Can Transfer to Allcited by 79
  3. 2021BACKDOORL: Backdoor Attack against Competitive Reinforcement Learningcited by 66
  4. 2020Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learningcited by 62
  5. 2021Adversarial Attacks On Multi-Agent Communicationcited by 48
  6. 2022Fine-Tuning Is All You Need to Mitigate Backdoor Attackscited by 35
  7. 2024Best-of-Venom: Attacking RLHF by Injecting Poisoned Preference Datacited by 33
  8. 2022Adversarial Policies Beat Superhuman Go AIscited by 29
  9. 2023BadRL: Sparse Targeted Backdoor Attack Against Reinforcement Learningcited by 29
  10. 2024Is poisoning a real threat to LLM alignment? Maybe more so than you thinkcited by 27
  11. 2021Reward Poisoning in Reinforcement Learning: Attacks Against Unknown Learners in Unknown Environmentscited by 26
  12. 2019Spatiotemporally Constrained Action Space Attacks on Deep Reinforcement Learning Agentscited by 24
  13. 2024Plasticity Loss in Deep Reinforcement Learning: A Surveycited by 24
  14. 2019Design of intentional backdoors in sequential modelscited by 20
  15. 2024SleeperNets: Universal Backdoor Poisoning Attacks Against Reinforcement Learning Agentscited by 19
  16. 2023Reinforcement Unlearningcited by 16
  17. 2023Implicit Poisoning Attacks in Two-Agent Reinforcement Learning: Adversarial Policies for Training-Time Attackscited by 14
  18. 2023ORL-AUDITOR: Dataset Auditing in Offline Deep Reinforcement Learningcited by 11
  19. 2023Rethinking Adversarial Policies: A Generalized Attack Formulation and Provable Defense in RLcited by 7
  20. 2024SUB-PLAY: Adversarial Policies against Partially Observed Multi-Agent Reinforcement Learning Systemscited by 7
  21. 2024Is Mamba Compatible with Trajectory Optimization in Offline Reinforcement Learning?cited by 7
  22. 2024Online Poisoning Attack Against Reinforcement Learning under Black-box Environmentscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2025TooBadRL: Trigger Optimization to Boost Effectiveness of Backdoor Attacks on Deep Reinforcement Learningcited by 2
  2. 2025TrojanTO: Action-Level Backdoor Attacks against Trajectory Optimization Modelscited by 2
  3. 2026Beware Untrusted Simulators -- Reward-Free Backdoor Attacks in Reinforcement Learningcited by 2
  4. 2026Angel or Demon: Investigating the Plasticity Interventions' Impact on Backdoor Threats in Deep Reinforcement Learningcited by 1
  5. 2026Downsides of Smartness Across Edge-Cloud Continuum in Modern Industrycited by 0