# Probabilistically safe and efficient model-based reinforcement learning

arXiv: https://arxiv.org/abs/2504.00626 · HTML: https://mlsanity.com/paper/2504.00626/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods.
- Builds on 14 of its 35 references that are in the map.
- First posted on arXiv 1 Apr 2025.

## Placed in

- Deep reinforcement learning methods / [Safe and constrained RL](https://mlsanity.com/deeprl/safe-and-constrained-rl/)

## Builds on (most cited first, 14 of 14)

- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [Control Barrier Functions: Theory and Applications](https://mlsanity.com/paper/1903.11199/) (2019)
- [Safe Reinforcement Learning Using Robust MPC](https://mlsanity.com/paper/1906.04005/) (2019)
- [Data-driven Economic NMPC using Reinforcement Learning](https://mlsanity.com/paper/1904.04152/) (2019)
- [Safety-Critical Model Predictive Control with Discrete-Time Control Barrier Function](https://mlsanity.com/paper/2007.11718/) (2020)
- [Synthesis of Model Predictive Control and Reinforcement Learning: Survey and Classification](https://mlsanity.com/paper/2502.02133/) (2025)
- [The Scenario Approach for Stochastic Model Predictive Control with Bounds on Closed-Loop Constraint Violations](https://mlsanity.com/paper/1307.5640/) (2013)
- [Practical Reinforcement Learning For MPC: Learning from sparse objectives in under an hour on a real robot](https://mlsanity.com/paper/2003.03200/) (2020)
- [Reinforcement Learning-based Receding Horizon Control using Adaptive Control Barrier Functions for Safety-Critical Systems](https://mlsanity.com/paper/2403.17338/) (2024)
- [Computing Probabilistic Controlled Invariant Sets](https://mlsanity.com/paper/1905.04117/) (2019)
- [Learning safety in model-based Reinforcement Learning using MPC and Gaussian Processes](https://mlsanity.com/paper/2211.01860/) (2022)
- [Probabilistically safe controllers based on control barrier functions and scenario model predictive control](https://mlsanity.com/paper/2409.06834/) (2024)
- [Approximate Dynamic Programming for Constrained Linear Systems: A Piecewise Quadratic Approximation Approach](https://mlsanity.com/paper/2205.10065/) (2022)
- [Parameterized Barrier Functions to Guarantee Safety under Uncertainty](https://mlsanity.com/paper/2303.11385/) (2023)

## Built on by (most cited first, 3 of 3)

- [Safety-Critical Reinforcement Learning with Viability-Based Action Shielding for Hypersonic Longitudinal Flight](https://mlsanity.com/paper/2602.03968/) (2026)
- [A Control Barrier Function-Constrained Model Predictive Control Framework for Safe Reinforcement Learning](https://mlsanity.com/paper/2604.06463/) (2026)
- [Learning-Based Stochastic Optimal Control with Infinite-Horizon Probabilistic Constraints](https://mlsanity.com/paper/2608.01151/) (2026)

## Abstract (arXiv)

This paper proposes tackling safety-critical stochastic Reinforcement Learning (RL) tasks with a sample-based, model-based approach. At the core of the method lies a Model Predictive Control (MPC) scheme that acts as function approximation, providing a model-based predictive control policy. To ensure safety, a probabilistic Control Barrier Function (CBF) is integrated into the MPC controller. To approximate the effects of stochasticies in the optimal control formulation and to fulfil the probabilistic CBF condition, a sample-based approach with guarantees is employed. Furthermore, to counterbalance the additional computational burden due to sampling, a learnable terminal cost formulation is included in the MPC objective. An RL algorithm is deployed to learn both the terminal cost and the CBF constraint. Results from a numerical experiment on a constrained LTI problem corroborate the effectiveness of the proposed methodology in reducing computation time while preserving control performance and safety.