mlsanityData to 14 Oct 2026

A Unified and Constrained View of Regularization-Based Robust Reinforcement Learning

arXiv 2609.13050 ↗First posted 11 Sep 2026cs.LGcited by 0

On the map: Deep reinforcement learning methods → Adversarial robustness and RL in finance

Abstract (arXiv)

Regularization-based methods have become a standard approach for training Deep Reinforcement Learning policies against adversarial input perturbations. In this paper, we unify these methods by deriving new upper bounds on the performance gap between the nominal and worst-case policies. Each upper bound is expressed as an existing regularization objective plus a KL-divergence penalty between the nominal and worst-case policies, which further explains why adding a KL penalty improves robustness in practice. Building on these bounds, we formulate robust training as a constrained optimization problem, showing that existing methods correspond to the special case of a fixed Lagrange multiplier. We instead update the multiplier jointly with the policy to automatically tune the regularization weight. Finally, we conduct extensive adversarial evaluations across several continuous control tasks to validate our theoretical analysis.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods. Builds on 10 of its 10 references that are in the map. First posted on arXiv 11 Sep 2026.

Builds on

10 of 10 references.

References · 10

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Trust Region Policy Optimizationcited by 2,580
  3. 2017Constrained Policy Optimizationcited by 983
  4. 2018Reward Constrained Policy Optimizationcited by 417
  5. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 316
  6. 2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 238
  7. 2021Robust Reinforcement Learning on State Observations with Learned Optimal Adversarycited by 135
  8. 2020Robust Deep Reinforcement Learning through Adversarial Losscited by 75
  9. 2021Who Is the Strongest Enemy? Towards Optimal and Efficient Evasion Attacks in Deep RLcited by 55
  10. 2022Efficient Adversarial Training without Attacking: Worst-Case-Aware Robust Reinforcement Learningcited by 43

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Robust Adversarial Reinforcement Learning with Risk Sensitivity and Critic Consistency Regularization
  2. 2025Viability of Future Actions: Robust Safety in Reinforcement Learning via Entropy Regularization
  3. 2026Provable Benefits of Regularization: Fast Rates for Adversarial Imitation Learning
  4. 2024Towards Optimal Adversarial Robust Q-learning with Bellman Infinity-error
  5. 2023A Connection between One-Step Regularization and Critic Regularization in Reinforcement Learning
  6. 2025Towards Optimal Adversarial Robust Reinforcement Learning with Infinity Measurement Error
  7. 2024Robust Offline Reinforcement Learning with Linearly Structured f-Divergence Regularization
  8. 2025State Entropy Regularization for Robust Reinforcement Learning