# Safe Reinforcement Learning with Probabilistic Control Barrier Functions for Ramp Merging

arXiv: https://arxiv.org/abs/2212.00618 · HTML: https://mlsanity.com/paper/2212.00618/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Safe offline RL, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 17 of its 32 references that are in the map.
- First posted on arXiv 1 Dec 2022.

## Placed in

- Offline RL and imitation learning from data / [Safe offline RL](https://mlsanity.com/offlinerl/safe-offline-rl/)

## Builds on (most cited first, 17 of 17)

- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Constrained Policy Optimization](https://mlsanity.com/paper/1705.10528/) (2017)
- [When to Trust Your Model: Model-Based Policy Optimization](https://mlsanity.com/paper/1906.08253/) (2019)
- [Control Barrier Functions: Theory and Applications](https://mlsanity.com/paper/1903.11199/) (2019)
- [Control Barrier Function Based Quadratic Programs for Safety Critical Systems](https://mlsanity.com/paper/1609.06408/) (2016)
- [Reinforcement Learning for Safety-Critical Control under Model Uncertainty, using Control Lyapunov Functions and Control Barrier Functions](https://mlsanity.com/paper/2004.07584/) (2020)
- [Safety-Critical Model Predictive Control with Discrete-Time Control Barrier Function](https://mlsanity.com/paper/2007.11718/) (2020)
- [Model-based Constrained Reinforcement Learning using Generalized Control Barrier Function](https://mlsanity.com/paper/2103.01556/) (2021)
- [Adaptive Safety with Control Barrier Functions](https://mlsanity.com/paper/1910.00555/) (2019)
- [Learning Robust Output Control Barrier Functions from Safe Expert Demonstrations](https://mlsanity.com/paper/2111.09971/) (2021)
- [Numerically Stable Dynamic Bicycle Model for Discrete-time Control](https://mlsanity.com/paper/2011.09612/) (2020)
- [Multi-Robot Collision Avoidance under Uncertainty with Probabilistic Safety Barrier Certificates](https://mlsanity.com/paper/1912.09957/) (2019)
- [Adaptive dynamic programming for nonaffine nonlinear optimal control problem with state constraints](https://mlsanity.com/paper/1911.11397/) (2019)
- [Probabilistic Safety-Assured Adaptive Merging Control for Autonomous Vehicles](https://mlsanity.com/paper/2104.14159/) (2021)
- [Probabilistic Safety Constraints for Learned High Relative Degree System Dynamics](https://mlsanity.com/paper/1912.10116/) (2019)
- [Feasibility Enhancement of Constrained Receding Horizon Control Using Generalized Control Barrier Function](https://mlsanity.com/paper/2102.13304/) (2021)

## Built on by (most cited first, 1 of 1)

- [A Survey on Physics Informed Reinforcement Learning: Review and Open Problems](https://mlsanity.com/paper/2309.01909/) (2023)

## Abstract (arXiv)

Prior work has looked at applying reinforcement learning and imitation learning approaches to autonomous driving scenarios, but either the safety or the efficiency of the algorithm is compromised. With the use of control barrier functions embedded into the reinforcement learning policy, we arrive at safe policies to optimize the performance of the autonomous driving vehicle. However, control barrier functions need a good approximation of the model of the car. We use probabilistic control barrier functions as an estimate of the model uncertainty. The algorithm is implemented as an online version in the CARLA (Dosovitskiy et al., 2017) Simulator and as an offline version on a dataset extracted from the NGSIM Database. The proposed algorithm is not just a safe ramp merging algorithm but a safe autonomous driving algorithm applied to address ramp merging on highways.