# Safe Exploration in Model-based Reinforcement Learning using Control Barrier Functions

arXiv: https://arxiv.org/abs/2104.08171 · HTML: https://mlsanity.com/paper/2104.08171/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 44 references that are in the map.
- First posted on arXiv 16 Apr 2021.

## Builds on (most cited first, 13 of 13)

- [Control Barrier Functions: Theory and Applications](https://mlsanity.com/paper/1903.11199/) (2019)
- [End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks](https://mlsanity.com/paper/1903.08792/) (2019)
- [Control Barrier Function Based Quadratic Programs for Safety Critical Systems](https://mlsanity.com/paper/1609.06408/) (2016)
- [A General Safety Framework for Learning-Based Control in Uncertain Robotic Systems](https://mlsanity.com/paper/1705.01292/) (2017)
- [Learning for Safety-Critical Control with Control Barrier Functions](https://mlsanity.com/paper/1912.10099/) (2019)
- [Data-driven Economic NMPC using Reinforcement Learning](https://mlsanity.com/paper/1904.04152/) (2019)
- [Robustness of Control Barrier Functions for Safety Critical Control](https://mlsanity.com/paper/1612.01554/) (2016)
- [Adaptive Safety with Control Barrier Functions](https://mlsanity.com/paper/1910.00555/) (2019)
- [Towards Robust Data-Driven Control Synthesis for Nonlinear Systems with Actuation Uncertainty](https://mlsanity.com/paper/2011.10730/) (2020)
- [Approximate Optimal Control for Safety-Critical Systems with Control Barrier Functions](https://mlsanity.com/paper/2008.04122/) (2020)
- [Concurrent learning-based approximate optimal regulation](https://mlsanity.com/paper/1304.3477/) (2013)
- [Efficient model-based reinforcement learning for approximate online optimal](https://mlsanity.com/paper/1502.02609/) (2015)
- [State Following (StaF) Kernel Functions for Function Approximation](https://mlsanity.com/paper/1503.04854/) (2015)

## Built on by (most cited first, 11 of 11)

- [A Survey on Physics Informed Reinforcement Learning: Review and Open Problems](https://mlsanity.com/paper/2309.01909/) (2023)
- [CBF-RL: Safety Filtering Reinforcement Learning in Training with Control Barrier Functions](https://mlsanity.com/paper/2510.14959/) (2025)
- [Approximation with Random Shallow ReLU Networks with Applications to Model Reference Adaptive Control](https://mlsanity.com/paper/2403.17142/) (2024)
- [A Review On Safe Reinforcement Learning Using Lyapunov and Barrier Functions](https://mlsanity.com/paper/2508.09128/) (2025)
- [Recursively Feasible Probabilistic Safe Online Learning with Control Barrier Functions](https://mlsanity.com/paper/2208.10733/) (2022)
- [Enhanced Safety in Autonomous Driving: Integrating Latent State Diffusion Model for End-to-End Navigation](https://mlsanity.com/paper/2407.06317/) (2024)
- [Function Gradient Approximation with Random Shallow ReLU Networks with Control Applications](https://mlsanity.com/paper/2410.05071/) (2024)
- [Safe Learning Control with Optimality and Stability Guarantees](https://mlsanity.com/paper/2501.15373/) (2025)
- [TRUST-UP: Trustworthy Reinforcement learning Using Safe Techniques for UAV Pursuit](https://mlsanity.com/paper/2411.17552/) (2024)
- [Reinforcement learning in pursuit-evasion differential game: safety, stability and robustness](https://mlsanity.com/paper/2507.19516/) (2025)
- [Game-Theoretic Risk-Shaped Reinforcement Learning for Safe Autonomous Driving](https://mlsanity.com/paper/2510.10960/) (2025)

## Abstract (arXiv)

This paper develops a model-based reinforcement learning (MBRL) framework for learning online the value function of an infinite-horizon optimal control problem while obeying safety constraints expressed as control barrier functions (CBFs). Our approach is facilitated by the development of a novel class of CBFs, termed Lyapunov-like CBFs (LCBFs), that retain the beneficial properties of CBFs for developing minimally-invasive safe control policies while also possessing desirable Lyapunov-like qualities such as positive semi-definiteness. We show how these LCBFs can be used to augment a learning-based control policy to guarantee safety and then leverage this approach to develop a safe exploration framework in a MBRL setting. We demonstrate that our approach can handle more general safety constraints than comparative methods via numerical examples.