mlsanityData to 7 Oct 2026

Robust Reinforcement Learning in Continuous Control Tasks with Uncertainty Set Regularization

arXiv 2207.02016 ↗First posted 5 Jul 2022cs.LG cs.AI cs.SYcited by 7

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning (RL) is recognized as lacking generalization and robustness under environmental perturbations, which excessively restricts its application for real-world robotics. Prior work claimed that adding regularization to the value function is equivalent to learning a robust policy with uncertain transitions. Although the regularization-robustness transformation is appealing for its simplicity and efficiency, it is still lacking in continuous control tasks. In this paper, we propose a new regularizer named Uncertainty Set Regularizer (USR), by formulating the uncertainty set on the parameter space of the transition function. In particular, USR is flexible enough to be plugged into any existing RL framework. To deal with unknown uncertainty sets, we further propose a novel adversarial approach to generate them based on the value function. We evaluate USR on the Real-world Reinforcement Learning (RWRL) benchmark, demonstrating improvements in the robust performance for perturbed testing environments.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 35 of its 68 references that are in the map. First posted on arXiv 5 Jul 2022.

Builds on

35 of 68 references.

References · 35

  1. 2013Intriguing properties of neural networkscited by 5,114
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  4. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  5. 2018DeepMind Control Suitecited by 935
  6. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
  7. 2018Learning Dexterous In-Hand Manipulationcited by 643
  8. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  9. 2017Robust Adversarial Reinforcement Learningcited by 439
  10. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 430
  11. 2016Learning to reinforcement learncited by 369
  12. 2019Challenges of Real-World Reinforcement Learningcited by 282
23 more
  1. 2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 235
  2. 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 169
  3. 2019Action Robust Reinforcement Learning and Applications in Continuous Controlcited by 167
  4. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  5. 2019Active Domain Randomizationcited by 161
  6. 2017Robust Deep Reinforcement Learning with Adversarial Attackscited by 152
  7. 2021Online Robust Reinforcement Learning with Model Uncertaintycited by 95
  8. 2019Robust Reinforcement Learning for Continuous Control with Model Misspecificationcited by 76
  9. 2020Robust Deep Reinforcement Learning through Adversarial Losscited by 73
  10. 2021Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformerscited by 71
  11. 2020Robust Reinforcement Learning using Adversarial Populationscited by 64
  12. 2017Reinforcement Learning under Model Mismatchcited by 52
  13. 2021Multi-Agent Reinforcement Learning for Active Voltage Control on Power Distribution Networkscited by 52
  14. 2021Twice regularized MDPs and the equivalence between robustness and regularizationcited by 46
  15. 2020Robust Reinforcement Learning via Adversarial training with Langevin Dynamicscited by 45
  16. 2020Data-efficient Domain Randomization with Bayesian Optimizationcited by 43
  17. 2018Soft-Robust Actor-Critic Policy-Gradientcited by 37
  18. 2021Regularized Policies are Reward Robustcited by 32
  19. 2019Robust Recovery Controller for a Quadrupedal Robot using Deep Reinforcement Learningcited by 30
  20. 2020Scalable First-Order Methods for Robust MDPscited by 22
  21. 2022Your Policy Regularizer is Secretly an Adversarycited by 21
  22. 2019Regularization Matters in Policy Optimizationcited by 13
  23. 2021Model-based micro-data reinforcement learning: what are the crucial model properties and which model to choose?cited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 2; 2025: 1; 2026: 2.

7 papers.

  1. 2023Robust multi-agent coordination via evolutionary generation of auxiliary adversarial attackerscited by 15
  2. 2023Twice Regularized Markov Decision Processes: The Equivalence between Robustness and Regularizationcited by 1
  3. 2026What Intermediate Layers Know: Detecting Jailbreaks from Entropy Dynamicscited by 1
  4. 2024Lipschitz-Regularized Critics Lead to Policy Robustness Against Transition Dynamics Uncertaintycited by 0
  5. 2024UDUC: An Uncertainty-driven Approach for Learning-based Robust Controlcited by 0
  6. 2025ADARL: Adaptive Low-Rank Structures for Robust Policy Learning under Uncertaintycited by 0
  7. 2026Beyond Coefficients: Forecast-Necessity Testing for Interpretable Causal Discovery in Nonlinear Time-Series Modelscited by 0