mlsanityData to 14 Oct 2026

TrojanTO: Action-Level Backdoor Attacks against Trajectory Optimization Models

arXiv 2506.12815 ↗First posted 15 Jun 2025cs.LGcited by 3

On the map: Offline RL and imitation learning from data → Offline RL algorithms

also in 1 more
Abstract (arXiv)

Recent advances in Trajectory Optimization (TO) models have achieved remarkable success in offline reinforcement learning. However, their vulnerabilities against backdoor attacks are poorly understood. We find that existing backdoor attacks in reinforcement learning are based on reward manipulation, which are largely ineffective against the TO model due to its inherent sequence modeling nature. Moreover, the complexities introduced by high-dimensional action spaces further compound the challenge of action manipulation. To address these gaps, we propose TrojanTO, the first action-level backdoor attack against TO models. TrojanTO employs alternating training to enhance the connection between triggers and target actions for attack effectiveness. To improve attack stealth, it utilizes precise poisoning via trajectory filtering for normal performance and batch poisoning for trigger consistency. Extensive evaluations demonstrate that TrojanTO effectively implants backdoor attacks across diverse tasks and attack objectives with a low attack budget (0.3% of trajectories). Furthermore, TrojanTO exhibits broad applicability to DT, GDT, and DC, underscoring its scalability across diverse TO model architectures.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 54 of its 66 references that are in the map. First posted on arXiv 15 Jun 2025.

Builds on

54 of 66 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024Doubly Mild Generalization for Offline Reinforcement Learning12 Nov
  2. 2024Offline Reinforcement Learning with OOD State Correction and OOD Action Suppression25 Oct
  3. 2024TrajDeleter: Enabling Trajectory Forgetting in Offline Reinforcement Learning Agents18 Apr
  4. 2023ORL-AUDITOR: Dataset Auditing in Offline Deep Reinforcement Learning6 Sep
  5. 2022BAFFLE: Hiding Backdoors in Offline Reinforcement Learning Datasets7 Oct
  6. 2022Reward Poisoning Attacks on Offline Multi-Agent Reinforcement Learning4 Jun
  7. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learning15 Apr
  8. 2019Policy Poisoning in Batch Reinforcement Learning and Control13 Oct

Other references · 46

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  4. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  5. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  6. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  7. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  8. 2022A Generalist Agentcited by 808
  9. 2017Deep Reinforcement Learning that Matterscited by 754
  10. 2021MetaFormer Is Actually What You Need for Visioncited by 524
  11. 2018Detecting Backdoor Attacks on Deep Neural Networks by Activation Clusteringcited by 456
  12. 2019Adversarial Policies: Attacking Deep Reinforcement Learningcited by 214
34 more
  1. 2021Backdoor Attacks on Pre-trained Models by Layerwise Weight Poisoningcited by 113
  2. 2020Rewriting a Deep Generative Modelcited by 107
  3. 2017Vulnerability of Deep Reinforcement Learning to Policy Induction Attackscited by 91
  4. 2021BACKDOORL: Backdoor Attack against Competitive Reinforcement Learningcited by 68
  5. 2021Handcrafted Backdoors in Deep Neural Networkscited by 63
  6. 2024Knowledge Mechanisms in Large Language Models: A Survey and Perspectivecited by 51
  7. 2022On Transforming Reinforcement Learning by Transformer: The Development Trajectorycited by 36
  8. 2024Best-of-Venom: Attacking RLHF by Injecting Poisoned Preference Datacited by 35
  9. 2019TrojDRL: Trojan Attacks on Deep Reinforcement Learning Agentscited by 30
  10. 2023BadRL: Sparse Targeted Backdoor Attack Against Reinforcement Learningcited by 30
  11. 2022Adversarial Policies Beat Superhuman Go AIscited by 29
  12. 2024Is poisoning a real threat to LLM alignment? Maybe more so than you thinkcited by 28
  13. 2023Decision ConvFormer: Local Filtering in MetaFormer is Sufficient for Decision Makingcited by 26
  14. 2019Spatiotemporally Constrained Action Space Attacks on Deep Reinforcement Learning Agentscited by 25
  15. 2017Boosting Adversarial Attacks with Momentumcited by 24
  16. 2019Design of intentional backdoors in sequential modelscited by 21
  17. 2021Poisoning Deep Reinforcement Learning Agents with In-Distribution Triggerscited by 20
  18. 2023Attacks in Adversarial Machine Learning: A Systematic Survey from the Life-cycle Perspectivecited by 20
  19. 2023Graph Decision Transformercited by 20
  20. 2022PolicyCleanse: Backdoor Detection and Mitigation in Reinforcement Learningcited by 19
  21. 2024SleeperNets: Universal Backdoor Poisoning Attacks Against Reinforcement Learning Agentscited by 19
  22. 2022Provable Defense against Backdoor Policies in Reinforcement Learningcited by 18
  23. 2023Is Imitation All You Need? Generalized Decision-Making with Dual-Phase Trainingcited by 18
  24. 2023Reinforcement Unlearningcited by 17
  25. 2024RAT: Adversarial Attacks on Deep Reinforcement Agents for Targeted Behaviorscited by 16
  26. 2023Efficient Adversarial Attacks on Online Multi-agent Reinforcement Learningcited by 10
  27. 2024SUB-PLAY: Adversarial Policies against Partially Observed Multi-Agent Reinforcement Learning Systemscited by 7
  28. 2024Is Mamba Compatible with Trajectory Optimization in Offline Reinforcement Learning?cited by 7
  29. 2025BLAST: A Stealthy Backdoor Leverage Attack against Cooperative Multi-Agent Deep Reinforcement Learning based Systemscited by 7
  30. 2024A Spatiotemporal Stealthy Backdoor Attack against Cooperative Multi-Agent Deep Reinforcement Learningcited by 6
  31. 2025UNIDOOR: A Universal Framework for Action-Level Backdoor Attacks in Deep Reinforcement Learningcited by 5
  32. 2025Machine Learning Models Have a Supply Chain Problemcited by 4
  33. 2024Backdoor in Seconds: Unlocking Vulnerabilities in Large Pre-trained Models via Model Editingcited by 3
  34. 2025TooBadRL: Trigger Optimization to Boost Effectiveness of Backdoor Attacks on Deep Reinforcement Learningcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Beware Untrusted Simulators -- Reward-Free Backdoor Attacks in Reinforcement Learningcited by 2
  2. 2026When the World Lies: Backdoor Attacks on Latent World Models for Downstream Controlcited by 0
  3. 2026Towards Merging-conditional Backdoor Unalignment against Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Exposing Vulnerabilities in RL: A Novel Stealthy Backdoor Attack through Reward Poisoning ↗
  2. 2026BehaviorGuard: Online Backdoor Defense for Deep Reinforcement Learning
  3. 2026UBA-ORL: Unlearning-Activated Backdoor Attacks on Offline Reinforcement Learning ↗
  4. 2026When Backdoors Meet Partial Observability: Attacking Real-World Reinforcement Learning
  5. 2025Backdoors in DRL: Four Environments Focusing on In-distribution Triggers
  6. 2025PNAct: Crafting Backdoor Attacks in Safe Reinforcement Learning
  7. 2026Plan2Cleanse: Test-Time Backdoor Defense via Monte-Carlo Planning in Deep Reinforcement Learning
  8. 2025Beyond Training-time Poisoning: Component-level and Post-training Backdoors in Deep Reinforcement Learning