mlsanityData to 14 Oct 2026

SafePG: Safe and Globally Optimal Reinforcement Learning with Hard Constraints

arXiv 2609.14076 ↗First posted 12 Sep 2026eess.SY cs.SYcited by 0

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

We present an optimal and convergent model-free policy gradient (PG) reinforcement learning (RL) framework for controlling nonlinear dynamical systems under hard safety constraints. We first construct a class of stochastic wrapper policies centered around a deterministic controller, thereby enabling exploration in unknown environments while preserving the underlying deterministic control structure. We then define a class of parameterized safe-by-construction control policies by truncating these stochastic policies onto hard safety constraints. We next establish, via measure-theoretic arguments, that the potentially nonconvex RL objective under the truncated policy class, as well as its policy gradients, are well-defined. We then develop a model-free PG algorithm based on stochastic gradient ascent that directly searches over these truncated policies and leverage gradient dominance to establish convergence and optimality guarantees. Finally, we validate this framework through simulations on a safe quadrotor navigation problem.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 12 of its 21 references that are in the map. First posted on arXiv 12 Sep 2026.

Builds on

12 of 21 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025CBF-RL: Safety Filtering Reinforcement Learning in Training with Control Barrier Functions16 Oct
  2. 2024Sampling-based Safe Reinforcement Learning for Nonlinear Dynamical Systems6 Mar

Other references · 10

  1. 2017Constrained Policy Optimizationcited by 983
  2. 2019Control Barrier Functions: Theory and Applicationscited by 372
  3. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 371
  4. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 289
  5. 2018Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulatorcited by 250
  6. 2019Safe Reinforcement Learning Using Robust MPCcited by 85
  7. 2017OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real Worldcited by 70
  8. 2024Learning Optimal Deterministic Policies with Stochastic Policy Gradientscited by 12
  9. 2023Reduced Policy Optimization for Continuous Control with Hard Constraintscited by 6
  10. 2023Safe Control Design through Risk-Tunable Control Barrier Functionscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Anytime Safe Reinforcement Learning
  2. 2025Off-Policy Reinforcement Learning with Anytime Safety Guarantees via Robust Safe Gradient Flow
  3. 2021Safe Continuous Control with Constrained Model-Based Policy Optimization
  4. 2026CAPSULE: Control-Theoretic Action Perturbations for Safe Uncertainty-Aware Reinforcement Learning
  5. 2026Sampling-Based Safe Reinforcement Learning
  6. 2026Learning over Forward-Invariant Policy Classes: Reinforcement Learning without Safety Concerns
  7. 2024A Dynamic Safety Shield for Safe and Efficient Reinforcement Learning of Navigation Tasks
  8. 2026CSPO: Constraint-Sensitive Policy Optimization for Safe Reinforcement Learning