mlsanityData to 14 Oct 2026

Robust In-Context Reinforcement Learning Under Reward Poisoning Attacks

arXiv 2506.06891 ↗First posted 7 Jun 2025cs.LG cs.CRcited by 0

On the map: Deep reinforcement learning methods → Adversarial robustness and RL in finance

Abstract (arXiv)

We study the corruption-robustness of in-context reinforcement learning (ICRL), focusing on the Decision-Pretrained Transformer (DPT, Lee et al., 2023). To address the challenge of reward poisoning attacks targeting the DPT, we propose a novel adversarial training framework, called Adversarially Trained DPT (AT-DPT). Our method simultaneously trains a population of attackers to minimize the true reward of the DPT by poisoning environment rewards, and a DPT model to infer optimal actions from the poisoned data. We evaluate the effectiveness of our approach against standard bandit algorithms, including robust baselines designed to handle reward contamination. Our results show that AT-DPT significantly outperforms them in bandit settings under a learned attacker, and generalizes to more complex environments such as adaptive attackers and MDPs. It shows promise in ICRL as a meta-RL approach to learning effective corruption-robust algorithms.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods. Builds on 65 of its 79 references that are in the map. First posted on arXiv 7 Jun 2025.

Builds on

65 of 79 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Adversarially Robust Decision Transformer25 Jul

Other references · 64

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  6. 2013Intriguing properties of neural networkscited by 5,155
  7. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  8. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  9. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  10. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,106
  11. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,055
  12. 2016Practical Black-Box Attacks against Machine Learningcited by 903
38 more
  1. 2022A Generalist Agentcited by 808
  2. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 729
  3. 2017Evasion Attacks against Machine Learning at Test Timecited by 689
  4. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 503
  5. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 327
  6. 2017Adversarial Attacks on Neural Network Policiescited by 320
  7. 2023Poisoning Web-Scale Training Datasets is Practicalcited by 290
  8. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 276
  9. 2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 238
  10. 2019VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learningcited by 217
  11. 2022Multi-Game Decision Transformerscited by 214
  12. 2022In-context Reinforcement Learning with Algorithm Distillationcited by 190
  13. 2017Tactics of Adversarial Attack on Deep Reinforcement Learning Agentscited by 166
  14. 2017Robust Deep Reinforcement Learning with Adversarial Attackscited by 154
  15. 2020Dynamic Backdoor Attacks Against Machine Learning Modelscited by 151
  16. 2022Prompting Decision Transformer for Few-Shot Policy Generalizationcited by 146
  17. 2023Supervised Pretraining Can Learn In-Context Reinforcement Learningcited by 139
  18. 2018Stochastic bandits robust to adversarial corruptionscited by 119
  19. 2023Structured State Space Models for In-Context Reinforcement Learningcited by 111
  20. 2023A Tutorial on Meta-Reinforcement Learningcited by 108
  21. 2022Robust Reinforcement Learning using Offline Datacited by 96
  22. 2020Adaptive Reward-Poisoning Attacks against Reinforcement Learningcited by 86
  23. 2022RORL: Robust Offline Reinforcement Learning via Conservative Smoothingcited by 84
  24. 2016Data Poisoning Attacks on Factorization-Based Collaborative Filteringcited by 83
  25. 2020Policy Teaching via Environment Poisoning: Training-time Adversarial Attacks against Reinforcement Learningcited by 81
  26. 2019Corruption-robust exploration in episodic reinforcement learningcited by 72
  27. 2019Policy Poisoning in Batch Reinforcement Learning and Controlcited by 70
  28. 2020Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learningcited by 63
  29. 2024Universal Vulnerabilities in Large Language Models: Backdoor Attacks for In-context Learningcited by 53
  30. 2021Achieving Near Instance-Optimality and Minimax-Optimality in Stochastic and Adversarial Linear Bandits Simultaneouslycited by 48
  31. 2021A Model Selection Approach for Corruption Robust Reinforcement Learningcited by 45
  32. 2020Vulnerability-Aware Poisoning Mechanism for Online RL with Unknown Dynamicscited by 38
  33. 2021Robust Stochastic Linear Contextual Bandits Under Adversarial Attackscited by 33
  34. 2019Attacking Vision-based Perception in End-to-End Autonomous Driving Modelscited by 31
  35. 2023Corruption-Robust Offline Reinforcement Learning with General Function Approximationcited by 31
  36. 2021Robust Policy Gradient against Strong Data Corruptioncited by 30
  37. 2024LLMs Are In-Context Bandit Reinforcement Learnerscited by 29
  38. 2023Emergence of In-Context Reinforcement Learning from Noise Distillationcited by 27

50 most cited of 64 All 64 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026When Can You Poison Rewards? A Tight Characterization of Reward Poisoning in Linear MDPs
  2. 2021Corruption-Robust Offline Reinforcement Learning
  3. 2026Learning to Attack: A Bandit Approach to Adversarial Context Poisoning
  4. 2026Reward-Preserving Attacks For Robust Reinforcement Learning
  5. 2024Robust Deep Reinforcement Learning Through Adversarial Attacks and Training : A Survey
  6. 2025Towards Robust Deep Reinforcement Learning against Environmental State Perturbation
  7. 2025Policy Disruption in Reinforcement Learning:Adversarial Attack with Large Language Models and Critical State Identification
  8. 2024Towards Robust Policy: Enhancing Offline Reinforcement Learning with Adversarial Attacks and Defenses