mlsanityData to 14 Oct 2026

PNAct: Crafting Backdoor Attacks in Safe Reinforcement Learning

arXiv 2507.00485 ↗First posted 1 Jul 2025cs.LG cs.AIcited by 2

On the map: Deep reinforcement learning methods → Adversarial robustness and RL in finance

Abstract (arXiv)

Reinforcement Learning (RL) is widely used in tasks where agents interact with an environment to maximize rewards. Building on this foundation, Safe Reinforcement Learning (Safe RL) incorporates a cost metric alongside the reward metric, ensuring that agents adhere to safety constraints during decision-making. In this paper, we identify that Safe RL is vulnerable to backdoor attacks, which can manipulate agents into performing unsafe actions. First, we introduce the relevant concepts and evaluation metrics for backdoor attacks in Safe RL. It is the first attack framework in the Safe RL field that involves both Positive and Negative Action sample (PNAct) is to implant backdoors, where positive action samples provide reference actions and negative action samples indicate actions to be avoided. We theoretically point out the properties of PNAct and design an attack algorithm. Finally, we conduct experiments to evaluate the effectiveness of our proposed backdoor attack framework, evaluating it with the established metrics. This paper highlights the potential risks associated with Safe RL and underscores the feasibility of such attacks. Our code and supplementary material are available at https://github.com/azure-123/PNAct.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods. Builds on 20 of its 32 references that are in the map. First posted on arXiv 1 Jul 2025.

Builds on

20 of 32 references.

References · 20

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  3. 2017Constrained Policy Optimizationcited by 983
  4. 2018Reward Constrained Policy Optimizationcited by 417
  5. 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 337
  6. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 316
  7. 2020Projection-Based Constrained Policy Optimizationcited by 203
  8. 2023Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmarkcited by 143
  9. 2019Benchmarking Batch Deep Reinforcement Learning Algorithmscited by 111
  10. 2022Constrained Variational Policy Optimization for Safe Reinforcement Learningcited by 71
  11. 2021BACKDOORL: Backdoor Attack against Competitive Reinforcement Learningcited by 68
  12. 2018Accelerated Primal-Dual Policy Optimization for Safe Reinforcement Learningcited by 62
8 more
  1. 2022Constrained Policy Optimization via Bayesian World Modelscited by 57
  2. 2022On the Robustness of Safe Reinforcement Learning under Observational Perturbationscited by 38
  3. 2021Context-Aware Safe Reinforcement Learning for Non-Stationary Environmentscited by 33
  4. 2023BadRL: Sparse Targeted Backdoor Attack Against Reinforcement Learningcited by 30
  5. 2022PolicyCleanse: Backdoor Detection and Mitigation in Reinforcement Learningcited by 19
  6. 2022RoMFAC: A robust mean-field actor-critic reinforcement learning against adversarial perturbations on statescited by 16
  7. 2023Partially Observable Mean Field Multi-Agent Reinforcement Learning Based on Graph-Attentioncited by 7
  8. 2023Enhancing the Robustness of QMIX against State-adversarial Attackscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Dataset Poisoning Attacks on Behavioral Cloning Policiescited by 0
  2. 2026Safe-RULE: Safe Reinforcement UnLEarningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Exposing Vulnerabilities in RL: A Novel Stealthy Backdoor Attack through Reward Poisoning ↗
  2. 2024SleeperNets: Universal Backdoor Poisoning Attacks Against Reinforcement Learning Agents
  3. 2026Beware Untrusted Simulators -- Reward-Free Backdoor Attacks in Reinforcement Learning
  4. 2025Fox in the Henhouse: Supply-Chain Backdoor Attacks Against Reinforcement Learning
  5. 2022BAFFLE: Hiding Backdoors in Offline Reinforcement Learning Datasets
  6. 2025UNIDOOR: A Universal Framework for Action-Level Backdoor Attacks in Deep Reinforcement Learning
  7. 2026PolicyGuard: Towards Test-time and Step-level Adversary (Backdoor) Defense for Reinforcement Learning Agent
  8. 2023Recover Triggered States: Protect Model Against Backdoor Attack in Reinforcement Learning