mlsanityData to 14 Oct 2026

Mitigating Deep Reinforcement Learning Backdoors in the Neural Activation Space

arXiv 2407.15168 ↗First posted 21 Jul 2024cs.LG cs.AI cs.CRcited by 4

On the map: Deep reinforcement learning methods → Adversarial robustness and RL in finance

Abstract (arXiv)

This paper investigates the threat of backdoors in Deep Reinforcement Learning (DRL) agent policies and proposes a novel method for their detection at runtime. Our study focuses on elusive in-distribution backdoor triggers. Such triggers are designed to induce a deviation in the behaviour of a backdoored agent while blending into the expected data distribution to evade detection. Through experiments conducted in the Atari Breakout environment, we demonstrate the limitations of current sanitisation methods when faced with such triggers and investigate why they present a challenging defence problem. We then evaluate the hypothesis that backdoor triggers might be easier to detect in the neural activation space of the DRL agent's policy network. Our statistical analysis shows that indeed the activation patterns in the agent's policy network are distinct in the presence of a trigger, regardless of how well the trigger is concealed in the environment. Based on this, we propose a new defence approach that uses a classifier trained on clean environment samples and detects abnormal activations. Our results show that even lightweight classifiers can effectively prevent malicious actions with considerable accuracy, indicating the potential of this research direction even against sophisticated adversaries.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods. Builds on 18 of its 31 references that are in the map. First posted on arXiv 21 Jul 2024.

Builds on

18 of 31 references.

References · 18

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  3. 2015Trust Region Policy Optimizationcited by 2,580
  4. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  5. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,055
  6. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  7. 2018Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networkscited by 519
  8. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 327
  9. 2017Tactics of Adversarial Attack on Deep Reinforcement Learning Agentscited by 166
  10. 2020Policy Teaching via Environment Poisoning: Training-time Adversarial Attacks against Reinforcement Learningcited by 81
  11. 2021BACKDOORL: Backdoor Attack against Competitive Reinforcement Learningcited by 68
  12. 2019Autonomous Penetration Testing using Reinforcement Learningcited by 33
6 more
  1. 2022Offline Reinforcement Learning for Safer Blood Glucose Control in People with Type 1 Diabetescited by 23
  2. 2021Poisoning Deep Reinforcement Learning Agents with In-Distribution Triggerscited by 20
  3. 2022PolicyCleanse: Backdoor Detection and Mitigation in Reinforcement Learningcited by 19
  4. 2022Provable Defense against Backdoor Policies in Reinforcement Learningcited by 18
  5. 2022A Temporal-Pattern Backdoor Attack to Deep Reinforcement Learningcited by 16
  6. 2022Execute Order 66: Targeted Data Poisoning for Reinforcement Learningcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025BLAST: A Stealthy Backdoor Leverage Attack against Cooperative Multi-Agent Deep Reinforcement Learning based Systemscited by 7
  2. 2025Beyond Training-time Poisoning: Component-level and Post-training Backdoors in Deep Reinforcement Learningcited by 1
  3. 2026BehaviorGuard: Online Backdoor Defense for Deep Reinforcement Learningcited by 0
  4. 2026Plan2Cleanse: Test-Time Backdoor Defense via Monte-Carlo Planning in Deep Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Backdoors in DRL: Four Environments Focusing on In-distribution Triggers
  2. 2026When Backdoors Meet Partial Observability: Attacking Real-World Reinforcement Learning
  3. 2025TooBadRL: Trigger Optimization to Boost Effectiveness of Backdoor Attacks on Deep Reinforcement Learning
  4. 2024Adversarial Inception Backdoor Attacks against Reinforcement Learning
  5. 2025Fox in the Henhouse: Supply-Chain Backdoor Attacks Against Reinforcement Learning
  6. 2023BadRL: Sparse Targeted Backdoor Attack Against Reinforcement Learning
  7. 2024A Spatiotemporal Stealthy Backdoor Attack against Cooperative Multi-Agent Deep Reinforcement Learning
  8. 2022BAFFLE: Hiding Backdoors in Offline Reinforcement Learning Datasets