Plan2Cleanse: Test-Time Backdoor Defense via Monte-Carlo Planning in Deep Reinforcement Learning

arXiv 2605.09638 ↗First posted 10 May 2026cs.LGcited by 0

On the map: Deep reinforcement learning methods → Adversarial robustness and RL in finance

Abstract (arXiv)

Ensuring the security of reinforcement learning (RL) models is critical, particularly when they are trained by third parties and deployed in real-world systems. Attackers can implant backdoors into these models, causing them to behave normally under typical conditions, but execute malicious behaviors when specific triggers are activated. In this work, we propose Plan2Cleanse, a test-time detection and mitigation framework that adapts Monte Carlo Tree Search to efficiently identify and neutralize RL backdoor attacks without requiring model retraining. Our approach recasts backdoor detection as a planning problem, enabling systematic exploration of temporally extended trigger sequences while maintaining black-box access to the target policy. By leveraging the detection results, Plan2Cleanse can further achieve efficient mitigation through tree-search preventive replanning. We evaluated our method in competitive MuJoCo environments, simulated O-RAN wireless networks, and Atari games. Plan2Cleanse achieves substantial improvements, increasing trigger detection success rates by more than 61.4 percentage points in stealthy O-RAN scenarios and improving win rates from 35% to 53% in competitive Humanoid environments. These results demonstrate the effectiveness of our test-time defense approach and highlight the importance of proactive defenses against backdoor threats in RL deployments. Our implementation is publicly available at https://github.com/rl-bandits-lab/RL-Backdoor.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods. Builds on 44 of its 63 references that are in the map. First posted on arXiv 10 May 2026.

Builds on

44 of 63 references.

Closest ancestors · 4

Same direction, newest first

Other references · 40

2017Proximal Policy Optimization Algorithmscited by 15,215
2013Playing Atari with Deep Reinforcement Learningcited by 3,274
2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
2022A Generalist Agentcited by 808
2017Deep Reinforcement Learning that Matterscited by 754
2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 580
2018Spectral Signatures in Backdoor Attackscited by 484
2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 459
2018Detecting Backdoor Attacks on Deep Neural Networks by Activation Clusteringcited by 456
2019Adversarial Policies: Attacking Deep Reinforcement Learningcited by 214

28 more

2018Applications of Deep Reinforcement Learning in Communications and Networking: A Surveycited by 167
2022Sionna: An Open-Source Library for Next-Generation Physical Layer Researchcited by 145
2021AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehiclescited by 137
2017Emergent Complexity via Multi-Agent Competitioncited by 120
2020The Ingredients of Real-World Robotic Reinforcement Learningcited by 117
2020Practical Detection of Trojan Neural Networks: Data-Limited and Data-Free Casescited by 95
2022Offline Q-Learning on Diverse Multi-Task Data Both Scales And Generalizescited by 73
2021BACKDOORL: Backdoor Attack against Competitive Reinforcement Learningcited by 68
2023Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learningcited by 54
2021AEVA: Black-box Backdoor Detection Using Adversarial Extreme Value Analysiscited by 53
2017Deep-Reinforcement Learning Multiple Access for Heterogeneous Wireless Networkscited by 37
2020Stop-and-Go: Exploring Backdoor Attacks on Deep Reinforcement Learning-based Traffic Congestion Control Systemscited by 37
2020Neural Bridge Sampling for Evaluating Safety-Critical Autonomous Systemscited by 35
2023BadRL: Sparse Targeted Backdoor Attack Against Reinforcement Learningcited by 30
2025Fast Adaptation with Behavioral Foundation Modelscited by 21
2022PolicyCleanse: Backdoor Detection and Mitigation in Reinforcement Learningcited by 19
2019Deep Reinforcement Learning for 5G Networks: Joint Beamforming, Power Control, and Interference Coordinationcited by 18
2022Provable Defense against Backdoor Policies in Reinforcement Learningcited by 18
2023PPO-Clip Attains Global Optimality: Towards Deeper Understandings of Clippingcited by 17
2022A Temporal-Pattern Backdoor Attack to Deep Reinforcement Learningcited by 16
2024A Method for Evaluating Hyperparameter Sensitivity in Reinforcement Learningcited by 15
2024System-level Analysis of Adversarial Attacks and Defenses on Intelligence in O-RAN based Cellular Networkscited by 8
2022Open RAN Security: Challenges and Opportunitiescited by 6
2024DRL-based Latency-Aware Network Slicing in O-RAN with Time-Varying SLAscited by 5
2022AI Testing Framework for Next-G O-RAN Networks: Requirements, Design, and Research Opportunitiescited by 4
2023Experimental Study of Adversarial Attacks on ML-based xApps in O-RANcited by 4
2019Adversarial ML Attack on Self Organizing Cellular Networkscited by 3
2025Test-Time Alignment for Large Language Models via Textual Model Predictive Controlcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way