# Model-based Constrained Reinforcement Learning using Generalized Control Barrier Function

arXiv: https://arxiv.org/abs/2103.01556 · HTML: https://mlsanity.com/paper/2103.01556/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 26 references that are in the map.
- First posted on arXiv 2 Mar 2021.

## Builds on (most cited first, 11 of 11)

- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Constrained Policy Optimization](https://mlsanity.com/paper/1705.10528/) (2017)
- [Reward Constrained Policy Optimization](https://mlsanity.com/paper/1805.11074/) (2018)
- [Control Barrier Functions: Theory and Applications](https://mlsanity.com/paper/1903.11199/) (2019)
- [Risk-Constrained Reinforcement Learning with Percentile Risk Criteria](https://mlsanity.com/paper/1512.01629/) (2015)
- [Learning-based Model Predictive Control for Safe Exploration](https://mlsanity.com/paper/1803.08287/) (2018)
- [Numerically Stable Dynamic Bicycle Model for Discrete-time Control](https://mlsanity.com/paper/2011.09612/) (2020)
- [Adaptive dynamic programming for nonaffine nonlinear optimal control problem with state constraints](https://mlsanity.com/paper/1911.11397/) (2019)
- [Improving Generalization of Reinforcement Learning with Minimax Distributional Soft Actor-Critic](https://mlsanity.com/paper/2002.05502/) (2020)
- [Feasibility Enhancement of Constrained Receding Horizon Control Using Generalized Control Barrier Function](https://mlsanity.com/paper/2102.13304/) (2021)

## Built on by (most cited first, 30 of 32)

- [Physics-Informed Machine Learning: A Survey on Problems, Methods and Applications](https://mlsanity.com/paper/2211.08064/) (2022)
- [Enforcing Hard Constraints with Soft Barriers: Safe Reinforcement Learning in Unknown Stochastic Environments](https://mlsanity.com/paper/2209.15090/) (2022)
- [Reachability Constrained Reinforcement Learning](https://mlsanity.com/paper/2205.07536/) (2022)
- [Enhance Sample Efficiency and Robustness of End-to-end Urban Autonomous Driving via Semantic Masked World Model](https://mlsanity.com/paper/2210.04017/) (2022)
- [Feasible Actor-Critic: Constrained Reinforcement Learning for Ensuring Statewise Safety](https://mlsanity.com/paper/2105.10682/) (2021)
- [Joint Synthesis of Safety Certificate and Safe Control Policy using Constrained Reinforcement Learning](https://mlsanity.com/paper/2111.07695/) (2021)
- [A Survey on Physics Informed Reinforcement Learning: Review and Open Problems](https://mlsanity.com/paper/2309.01909/) (2023)
- [CBF-RL: Safety Filtering Reinforcement Learning in Training with Control Barrier Functions](https://mlsanity.com/paper/2510.14959/) (2025)
- [Hamilton-Jacobi Reachability in Reinforcement Learning: A Survey](https://mlsanity.com/paper/2407.09645/) (2024)
- [Optimal Transport Perturbations for Safe Reinforcement Learning with Robustness Guarantees](https://mlsanity.com/paper/2301.13375/) (2023)
- [Sequential Neural Barriers for Scalable Dynamic Obstacle Avoidance](https://mlsanity.com/paper/2307.03015/) (2023)
- [Learning Control Barrier Functions and their application in Reinforcement Learning: A Survey](https://mlsanity.com/paper/2404.16879/) (2024)
- [Gaussian Max-Value Entropy Search for Multi-Agent Bayesian Optimization](https://mlsanity.com/paper/2303.05694/) (2023)
- [How Does the Lagrangian Guide Safe Reinforcement Learning through Diffusion Models?](https://mlsanity.com/paper/2602.02924/) (2026)
- [Learn Zero-Constraint-Violation Policy in Model-Free Constrained Reinforcement Learning](https://mlsanity.com/paper/2111.12953/) (2021)
- [Model-Based Safe Reinforcement Learning with Time-Varying State and Control Constraints: An Application to Intelligent Vehicles](https://mlsanity.com/paper/2112.11217/) (2021)
- [Safe and Sample-efficient Reinforcement Learning for Clustered Dynamic Environments](https://mlsanity.com/paper/2303.14265/) (2023)
- [Feasible Policy Iteration for Safe Reinforcement Learning](https://mlsanity.com/paper/2304.08845/) (2023)
- [Sampling-based Safe Reinforcement Learning for Nonlinear Dynamical Systems](https://mlsanity.com/paper/2403.04007/) (2024)
- [A Review On Safe Reinforcement Learning Using Lyapunov and Barrier Functions](https://mlsanity.com/paper/2508.09128/) (2025)
- [Model Free Barrier Functions via Implicit Evading Maneuvers](https://mlsanity.com/paper/2107.12871/) (2021)
- [Safe Inverse Reinforcement Learning via Control Barrier Function](https://mlsanity.com/paper/2212.02753/) (2022)
- [Improve Generalization of Driving Policy at Signalized Intersections with Adversarial Learning](https://mlsanity.com/paper/2204.04403/) (2022)
- [Control invariant set enhanced safe reinforcement learning: improved sampling efficiency, guaranteed stability and robustness](https://mlsanity.com/paper/2305.15602/) (2023)
- [Safe Reinforcement Learning with Probabilistic Control Barrier Functions for Ramp Merging](https://mlsanity.com/paper/2212.00618/) (2022)
- [The Feasibility Theory of Constrained Reinforcement Learning: A Tutorial Study](https://mlsanity.com/paper/2404.10064/) (2024)
- [Act Better by Timing: A timing-Aware Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/2406.13223/) (2024)
- [Augmented Lagrangian Multiplier Network for State-wise Safety in Reinforcement Learning](https://mlsanity.com/paper/2605.00667/) (2026)
- [Interpreting Primal-Dual Algorithms for Constrained Multiagent Reinforcement Learning](https://mlsanity.com/paper/2211.16069/) (2022)
- [Ensemble RL through Classifier Models: Enhancing Risk-Return Trade-offs in Trading Strategies](https://mlsanity.com/paper/2502.17518/) (2025)

## Abstract (arXiv)

Model information can be used to predict future trajectories, so it has huge potential to avoid dangerous region when implementing reinforcement learning (RL) on real-world tasks, like autonomous driving. However, existing studies mostly use model-free constrained RL, which causes inevitable constraint violations. This paper proposes a model-based feasibility enhancement technique of constrained RL, which enhances the feasibility of policy using generalized control barrier function (GCBF) defined on the distance to constraint boundary. By using the model information, the policy can be optimized safely without violating actual safety constraints, and the sample efficiency is increased. The major difficulty of infeasibility in solving the constrained policy gradient is handled by an adaptive coefficient mechanism. We evaluate the proposed method in both simulations and real vehicle experiments in a complex autonomous driving collision avoidance task. The proposed method achieves up to four times fewer constraint violations and converges 3.36 times faster than baseline constrained RL approaches.