mlsanityData to 14 Oct 2026

Robust Peak-cost Constrained Reinforcement Learning

arXiv 2607.15457 ↗First posted 16 Jul 2026cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

We study robust peak-cost constrained reinforcement learning (ours), where the objective is to maximize expected reward while controlling the maximum cost encountered along a trajectory. This setting is motivated by safety-critical applications in which a single large violation can be catastrophic and therefore cannot be adequately captured by the standard CMDP framework based on expected cumulative cost. Existing reachability-constrained RL methods adopt Lagrangian-based approaches, yet the underlying duality properties of peak-cost constrained MDPs remain unclear. We show that, unlike standard CMDPs, peak-cost constrained MDPs may not admit zero duality gap. We further consider a robust formulation to address simulator-to-real-world mismatch in the transition dynamics. To solve this problem, we develop a surrogate optimization framework and a robust value estimation method based on integral probability metrics. We prove that, with appropriate hyperparameter choices, the surrogate solution attains the same robust reward value as the original problem while violating the constraint by at most (ε). Experiments show that the proposed method effectively enforces safety under dynamics perturbations while retaining strong reward performance across diverse environments.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 24 of its 46 references that are in the map. First posted on arXiv 16 Jul 2026.

Builds on

24 of 46 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Efficient Policy Optimization in Robust Constrained MDPs with Iteration Complexity Guarantees25 May
  2. 2024Solving Minimum-Cost Reach Avoid using Reinforcement Learning29 Oct
  3. 2024Near-Optimal Policy Identification in Robust Constrained Markov Decision Processes via Epigraph Form29 Aug
  4. 2024Distributionally Robust Constrained Reinforcement Learning under Strong Duality22 Jun

Other references · 20

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2016OpenAI Gymcited by 1,736
  3. 2017Constrained Policy Optimizationcited by 983
  4. 2018Reward Constrained Policy Optimizationcited by 417
  5. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 371
  6. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 316
  7. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 297
  8. 2017Hamilton-Jacobi Reachability: A Brief Overview and Recent Advancescited by 201
  9. 2017A General Safety Framework for Learning-Based Control in Uncertain Robotic Systemscited by 163
  10. 2019Constrained Reinforcement Learning Has Zero Duality Gapcited by 151
  11. 2020CRPO: A New Approach for Safe Reinforcement Learning with Convergence Guaranteecited by 134
  12. 2019Safe Policies for Reinforcement Learning via Primal-Dual Methodscited by 79
8 more
  1. 2022Reachability Constrained Reinforcement Learningcited by 59
  2. 2021Robust Control Barrier-Value Functions for Safety-Critical Controlcited by 51
  3. 2020Constrained Upper Confidence Reinforcement Learningcited by 46
  4. 2021Learning Barrier Certificates: Towards Safe Reinforcement Learning with Zero Training-time Violationscited by 39
  5. 2023Iterative Reachability Estimation for Safe Reinforcement Learningcited by 37
  6. 2023Natural Actor-Critic for Robust Reinforcement Learning with Function Approximationcited by 36
  7. 2021Joint Synthesis of Safety Certificate and Safe Control Policy using Constrained Reinforcement Learningcited by 27
  8. 2022Robust Constrained Reinforcement Learningcited by 15

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Provably Efficient Sample Complexity for Robust CMDP
  2. 2024Time-Constrained Robust MDPs
  3. 2024Constrained Reinforcement Learning Under Model Mismatch
  4. 2026Beyond Hard Constraints: Budget-Conditioned Reachability For Safe Offline Reinforcement Learning
  5. 2025Risk-Averse Constrained Reinforcement Learning with Optimized Certainty Equivalents
  6. 2025Rectified Robust Policy Optimization for Model-Uncertain Constrained Reinforcement Learning without Strong Duality
  7. 2025Ensuring Safety in an Uncertain Environment: Constrained MDPs via Stochastic Thresholds
  8. 2026CSPO: Constraint-Sensitive Policy Optimization for Safe Reinforcement Learning