mlsanityData to 14 Oct 2026

CAPSULE: Control-Theoretic Action Perturbations for Safe Uncertainty-Aware Reinforcement Learning

arXiv 2604.23576 ↗First posted 26 Apr 2026cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

Ensuring safe exploration in high-dimensional systems with unknown dynamics remains a significant challenge. Existing safe reinforcement learning methods often provide safety guarantees only in expectation, which can still lead to safety violations. Control-theoretic approaches, in contrast, offer hard constraint-based safety guarantees but typically assume access to known system dynamics or require accurate estimation of control-affine models. In this paper, we propose a safe reinforcement learning framework that learns a probabilistic control-affine dynamics model in an offline setting. The learned model is leveraged to explicitly construct control barrier functions (CBFs) that incorporate model uncertainty to provide conservative safety constraints. These CBF constraints are enforced through an online constraint-based action correction mechanism, enabling safe exploration without overly restricting task performance. Empirical evaluations on nonlinear, complex continuous-control benchmarks demonstrate that our approach achieves returns comparable to those of existing baselines while significantly reducing safety violations.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 7 of its 16 references that are in the map. First posted on arXiv 26 Apr 2026.

Builds on

7 of 16 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Constrained Reinforcement Learning with Smoothed Log Barrier Function21 Mar

Other references · 6

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Constrained Policy Optimizationcited by 983
  3. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  4. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 289
  5. 2016Control Barrier Function Based Quadratic Programs for Safety Critical Systemscited by 287
  6. 2020Projection-Based Constrained Policy Optimizationcited by 203

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Sampling-Based Safe Reinforcement Learning
  2. 2021Safe Continuous Control with Constrained Model-Based Policy Optimization
  3. 2026Certifiable Safe Model-Based Reinforcement Learning with Control-Affine Dynamics Approximation
  4. 2024Handling Long-Term Safety and Uncertainty in Safe Reinforcement Learning
  5. 2024Sampling-based Safe Reinforcement Learning for Nonlinear Dynamical Systems
  6. 2026Dyna-Style Safety Augmented Reinforcement Learning: Staying Safe in the Face of Uncertainty
  7. 2026A Control Barrier Function-Constrained Model Predictive Control Framework for Safe Reinforcement Learning
  8. 2026SafePG: Safe and Globally Optimal Reinforcement Learning with Hard Constraints