mlsanityData to 7 Oct 2026

Distributionally Robust Constrained Reinforcement Learning under Strong Duality

arXiv 2406.15788 ↗First posted 22 Jun 2024cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

We study the problem of Distributionally Robust Constrained RL (DRC-RL), where the goal is to maximize the expected reward subject to environmental distribution shifts and constraints. This setting captures situations where training and testing environments differ, and policies must satisfy constraints motivated by safety or limited budgets. Despite significant progress toward algorithm design for the separate problems of distributionally robust RL and constrained RL, there do not yet exist algorithms with end-to-end convergence guarantees for DRC-RL. We develop an algorithmic framework based on strong duality that enables the first efficient and provable solution in a class of environmental uncertainties. Further, our framework exposes an inherent structure of DRC-RL that arises from the combination of distributional robustness and constraints, which prevents a popular class of iterative methods from tractably solving DRC-RL, despite such frameworks being applicable for each of distributionally robust RL and constrained RL individually. Finally, we conduct experiments on a car racing benchmark to evaluate the effectiveness of the proposed algorithm.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 37 of its 69 references that are in the map. First posted on arXiv 22 Jun 2024.

Builds on

37 of 69 references.

References · 37

  1. 2021Convex optimizationcited by 2,479
  2. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
  3. 2019A Theory of Regularized Markov Decision Processescited by 279
  4. 2019Batch Policy Learning under Constraintscited by 242
  5. 2020Exploration-Exploitation in Constrained MDPscited by 146
  6. 2019Constrained Reinforcement Learning Has Zero Duality Gapcited by 144
  7. 2020Provably Efficient Safe Exploration via Primal-Dual Policy Optimizationcited by 131
  8. 2020Safe Reinforcement Learning in Constrained Markov Decision Processescited by 109
  9. 2021Online Robust Reinforcement Learning with Model Uncertaintycited by 95
  10. 2022Robust Reinforcement Learning using Offline Datacited by 95
  11. 2022Distributionally Robust Model-Based Offline Reinforcement Learning with Near-Optimal Sample Complexitycited by 85
  12. 2021Sample Complexity of Robust Reinforcement Learning with a Generative Modelcited by 82
25 more
  1. 2019Robust Reinforcement Learning for Continuous Control with Model Misspecificationcited by 76
  2. 2022Policy Gradient Method For Robust Reinforcement Learningcited by 67
  3. 2022Saute RL: Almost Surely Safe Reinforcement Learning Using State Augmentationcited by 63
  4. 2019Reinforcement Learning with Convex Constraintscited by 57
  5. 2023The Curious Price of Distributional Robustness in Reinforcement Learning with a Generative Modelcited by 50
  6. 2021Twice regularized MDPs and the equivalence between robustness and regularizationcited by 46
  7. 2023Improved Sample Complexity Bounds for Distributionally Robust Reinforcement Learningcited by 44
  8. 2020Cautious Adaptation For Reinforcement Learning in Safety-Critical Settingscited by 42
  9. 2019Beyond Confidence Regions: Tight Bayesian Ambiguity Sets for Robust MDPscited by 37
  10. 2021How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Reviewcited by 36
  11. 2021DOPE: Doubly Optimistic and Pessimistic Exploration for Safe Reinforcement Learningcited by 34
  12. 2015Distributionally Robust Counterpart in Markov Decision Processescited by 32
  13. 2022First-order Policy Optimization for Robust Markov Decision Processcited by 31
  14. 2021Towards Theoretical Understandings of Robust Markov Decision Processes: Sample Complexity and Asymptoticscited by 27
  15. 2024Sample-Efficient Robust Multi-Agent Reinforcement Learning in the Face of Environmental Uncertaintycited by 24
  16. 2020Robust Constrained-MDPs: Soft-Constrained Robust Policy Optimization under Model Uncertaintycited by 23
  17. 2023Towards Minimax Optimality of Model-based Robust Reinforcement Learningcited by 21
  18. 2023Trust Region-Based Safe Distributional Reinforcement Learning for Multiple Constraintscited by 20
  19. 2022Safe Reinforcement Learning of Dynamic High-Dimensional Robotic Tasks: Navigation, Manipulation, Interactioncited by 15
  20. 2022Robust Constrained Reinforcement Learningcited by 14
  21. 2023Risk-Averse Model Uncertainty for Distributionally Robust Safe Reinforcement Learningcited by 12
  22. 2021Safe Policy Optimization with Local Generalized Linear Function Approximationscited by 10
  23. 2023First-order Policy Optimization for Robust Policy Evaluationcited by 6
  24. 2024Constrained Reinforcement Learning Under Model Mismatchcited by 6
  25. 2023Robust Lagrangian and Adversarial Policy Gradient for Robust Constrained Markov Decision Processescited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 10; 2026: 4.

15 papers.

Sort:

By area:

  1. 2025Robust Gymnasium: A Unified Modular Benchmark for Robust Reinforcement Learningcited by 12
  2. 2024Near-Optimal Policy Identification in Robust Constrained Markov Decision Processes via Epigraph Formcited by 10
  3. 2025MOBODY: Model Based Off-Dynamics Offline Reinforcement Learningcited by 8
  4. 2025SPiDR: A Simple Approach for Zero-Shot Safety in Sim-to-Real Transfercited by 8
  5. 2025Efficient Policy Optimization in Robust Constrained MDPs with Iteration Complexity Guaranteescited by 6
  6. 2026Safe Exploration via Policy Priorscited by 6
  7. 2025Rectified Robust Policy Optimization for Model-Uncertain Constrained Reinforcement Learning without Strong Dualitycited by 4
  8. 2025Distributionally Robust Multi-Agent Reinforcement Learning for Dynamic Chute Mappingcited by 2
  9. 2025Primal-Only Actor Critic Algorithm for Robust Constrained Average Cost MDPscited by 1
  10. 2025Provably Efficient Sample Complexity for Robust CMDPcited by 1
  11. 2025Temporal Robustness in Discrete Time Linear Dynamical Systemscited by 0
  12. 2025Initial Distribution Sensitivity of Constrained Markov Decision Processescited by 0
3 more
  1. 2026Distributionally Robust Cooperative Multi-Agent Reinforcement Learning via Robust Value Factorizationcited by 0
  2. 2026Robust Peak-cost Constrained Reinforcement Learningcited by 0
  3. 2026Provably Safe Sim-to-Real Transfercited by 0