Lyapunov Robust Constrained-MDPs: Soft-Constrained Robustly Stable Policy Optimization under Model Uncertainty
Not on the map. Why?
Abstract (arXiv)
Safety and robustness are two desired properties for any reinforcement learning algorithm. CMDPs can handle additional safety constraints and RMDPs can perform well under model uncertainties. In this paper, we propose to unite these two frameworks resulting in robust constrained MDPs (RCMDPs). The motivation is to develop a framework that can satisfy safety constraints while also simultaneously offer robustness to model uncertainties. We develop the RCMDP objective, derive gradient update formula to optimize this objective and then propose policy gradient based algorithms. We also independently propose Lyapunov based reward shaping for RCMDPs, yielding better stability and convergence properties.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 11 of its 34 references that are in the map. First posted on arXiv 5 Aug 2021.
Builds on
11 of 34 references.
References · 11
- 2017Constrained Policy Optimizationcited by 928
- 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
- 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 278
- 2018Safe Exploration in Continuous Action Spacescited by 237
- 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 225
- 2019Constrained Reinforcement Learning Has Zero Duality Gapcited by 144
- 2014Optimizing the CVaR via Samplingcited by 98
- 2014Algorithms for CVaR Optimization in MDPscited by 98
- 2018Soft-Robust Actor-Critic Policy-Gradientcited by 37
- 2019Beyond Confidence Regions: Tight Bayesian Ambiguity Sets for Robust MDPscited by 37
- 2018Sim-to-Real Transfer Learning using Robustified Controllers in Robotic Tasks involving Complex Dynamicscited by 7
Built on by
Citing papers per year: 2021: 1; 2022: 1; 2023: 1.
3 papers.