mlsanityData to 14 Oct 2026

Reward-Preserving Attacks For Robust Reinforcement Learning

arXiv 2601.07118 ↗First posted 12 Jan 2026cs.LGcited by 0

On the map: Deep reinforcement learning methods → Adversarial robustness and RL in finance

Abstract (arXiv)

Adversarial training in reinforcement learning (RL) is challenging because perturbations cascade through trajectories and compound over time, making fixed-strength attacks either overly destructive or too conservative. We propose reward-preserving attacks, which adapt adversarial strength so that an α fraction of the nominal-to-worst-case return gap remains achievable at each state. In deep RL, perturbation magnitudes η are selected dynamically, using a learned critic Q((s,a),η) that estimates the expected return of α-reward-preserving rollouts. For intermediate values of α, this adaptive training yields policies that are robust across a wide range of perturbation magnitudes while preserving nominal performance, outperforming fixed-radius and uniformly sampled-radius adversarial training.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods. Builds on 17 of its 26 references that are in the map. First posted on arXiv 12 Jan 2026.

Builds on

17 of 26 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Robust Deep Reinforcement Learning with Adaptive Adversarial Perturbations in Action Space20 May
  2. 2024Robust Deep Reinforcement Learning Through Adversarial Attacks and Training : A Survey1 Mar

Other references · 15

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  5. 2015Trust Region Policy Optimizationcited by 2,580
  6. 2017Robust Adversarial Reinforcement Learningcited by 451
  7. 2017Adversarial Attacks on Neural Network Policiescited by 320
  8. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 275
  9. 2019Active Domain Randomizationcited by 170
  10. 2021Robust Reinforcement Learning on State Observations with Learned Optimal Adversarycited by 135
  11. 2019Wasserstein Robust Reinforcement Learningcited by 54
  12. 2019Improved Robustness and Safety for Autonomous Vehicle Control with Adversarial Reinforcement Learningcited by 23
3 more
  1. 2021Distributionally Robust Policy Learning via Adversarial Environment Generationcited by 17
  2. 2022Max-Min Off-Policy Actor-Critic Method Focusing on Worst-Case Robustness to Model Misspecificationcited by 11
  3. 2024Time-Constrained Robust MDPscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Beyond Worst-case Attacks: Robust RL with Adaptive Defense via Non-dominated Policies
  2. 2026Robust Adversarial Reinforcement Learning with Risk Sensitivity and Critic Consistency Regularization
  3. 2026Taming the Adversary: A Cost-to-Disturbance Ratio Approach to Adversarial Reinforcement Learning
  4. 2025Towards Robust Deep Reinforcement Learning against Environmental State Perturbation
  5. 2024Towards Robust Policy: Enhancing Offline Reinforcement Learning with Adversarial Attacks and Defenses
  6. 2020oIRL: Robust Adversarial Inverse Reinforcement Learning with Temporally Extended Actions
  7. 2024Training Verifiably Robust Agents Using Set-Based Reinforcement Learning
  8. 2025Adversarial Diffusion for Robust Reinforcement Learning