# CBF-RL: Safety Filtering Reinforcement Learning in Training with Control Barrier Functions arXiv: https://arxiv.org/abs/2510.14959 ยท HTML: https://mlsanity.com/paper/2510.14959/ ## Key facts - Cited by 17 papers in the map (data through 14 Oct 2026). - Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. - Builds on 30 of its 38 references that are in the map. - First posted on arXiv 16 Oct 2025. ## Placed in - Deep reinforcement learning methods / [Safe and constrained RL](https://mlsanity.com/deeprl/safe-and-constrained-rl/) ## Builds on (most cited first, 30 of 30) - [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017) - [Safe Reinforcement Learning via Shielding](https://mlsanity.com/paper/1708.08611/) (2017) - [Control Barrier Functions: Theory and Applications](https://mlsanity.com/paper/1903.11199/) (2019) - [Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments](https://mlsanity.com/paper/2301.04195/) (2023) - [End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks](https://mlsanity.com/paper/1903.08792/) (2019) - [Control Barrier Function Based Quadratic Programs for Safety Critical Systems](https://mlsanity.com/paper/1609.06408/) (2016) - [Asymmetric Actor Critic for Image-Based Robot Learning](https://mlsanity.com/paper/1710.06542/) (2017) - [BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusion](https://mlsanity.com/paper/2508.08241/) (2025) - [ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills](https://mlsanity.com/paper/2502.01143/) (2025) - [A predictive safety filter for learning-based control of constrained nonlinear dynamical systems](https://mlsanity.com/paper/1812.05506/) (2018) - [Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control](https://mlsanity.com/paper/2401.16889/) (2024) - [Visual Imitation Enables Contextual Humanoid Control](https://mlsanity.com/paper/2505.03729/) (2025) - [Safe Reinforcement Learning Using Robust Control Barrier Functions](https://mlsanity.com/paper/2110.05415/) (2021) - [Robust Model Predictive Shielding for Safe Reinforcement Learning with Stochastic Dynamics](https://mlsanity.com/paper/1910.10885/) (2019) - [Provably Safe Reinforcement Learning: Conceptual Analysis, Survey, and Benchmarking](https://mlsanity.com/paper/2205.06750/) (2022) - [HITTER: A HumanoId Table TEnnis Robot via Hierarchical Planning and Learning](https://mlsanity.com/paper/2508.21043/) (2025) - [Model-based Constrained Reinforcement Learning using Generalized Control Barrier Function](https://mlsanity.com/paper/2103.01556/) (2021) - [Safety Filtering While Training: Improving the Performance and Sample Efficiency of Reinforcement Learning Agents](https://mlsanity.com/paper/2410.11671/) (2024) - [Reinforcement Learning for Safe Robot Control using Control Lyapunov Barrier Functions](https://mlsanity.com/paper/2305.09793/) (2023) - [Safe Policy Synthesis in Multi-Agent POMDPs via Discrete-Time Barrier Functions](https://mlsanity.com/paper/1903.07823/) (2019) - [Safe Exploration in Model-based Reinforcement Learning using Control Barrier Functions](https://mlsanity.com/paper/2104.08171/) (2021) - [Safe and Efficient Reinforcement Learning Using Disturbance-Observer-Based Control Barrier Functions](https://mlsanity.com/paper/2211.17250/) (2022) - [Barrier Functions Inspired Reward Shaping for Reinforcement Learning](https://mlsanity.com/paper/2403.01410/) (2024) - [Omni-Perception: Omnidirectional Collision Avoidance for Legged Locomotion in Dynamic Environments](https://mlsanity.com/paper/2505.19214/) (2025) - [Control Barrier Functions in Sampled-Data Systems](https://mlsanity.com/paper/2103.03677/) (2021) - [Value Functions are Control Barrier Functions: Verification of Safe Policies using Control Theory](https://mlsanity.com/paper/2306.04026/) (2023) - [Gait-Conditioned Reinforcement Learning with Multi-Phase Curriculum for Humanoid Locomotion](https://mlsanity.com/paper/2505.20619/) (2025) - [Optimizing Bipedal Locomotion for The 100m Dash With Comparison to Human Running](https://mlsanity.com/paper/2508.03070/) (2025) - [Safety-Critical Controller Synthesis with Reduced-Order Models](https://mlsanity.com/paper/2411.16479/) (2024) - [Safe Reinforcement Learning for an Energy-Efficient Driver Assistance System](https://mlsanity.com/paper/2301.00904/) (2023) ## Built on by (most cited first, 17 of 17) - [LIBERO-Safety: A Comprehensive Benchmark for Physical and Semantic Safety in Vision-Language-Action Models](https://mlsanity.com/paper/2606.23686/) (2026) - [PAC-MAN: Perception-Aware CBF-RL for Whole-Body Safety in Humanoid Dodgeball](https://mlsanity.com/paper/2607.28623/) (2026) - [HANDOFF: Humanoid Agentic Task-Space Whole-Body Control via Distilled Complementary Teachers](https://mlsanity.com/paper/2606.06493/) (2026) - [Provably Safe, Yet Scalable Reinforcement Learning](https://mlsanity.com/paper/2606.14536/) (2026) - [MPC-Injection: Biasing Off-Policy Locomotion RL Toward Controller-Induced Behavior Basins](https://mlsanity.com/paper/2606.26392/) (2026) - [Model-free LQG Control with Chance Constraints](https://mlsanity.com/paper/2605.31310/) (2026) - [Constrained Whole-Body Tracking for Humanoid Robots](https://mlsanity.com/paper/2606.00374/) (2026) - [World-Task Factorization for Robot Learning](https://mlsanity.com/paper/2606.02027/) (2026) - [Discriminative Barrier Functions for Safe Adversarial Imitation Learning from Observation](https://mlsanity.com/paper/2607.13938/) (2026) - [SafePG: Safe and Globally Optimal Reinforcement Learning with Hard Constraints](https://mlsanity.com/paper/2609.14076/) (2026) - [ResSafe: Learning Safety Filtering with Residual Reinforcement Learning for Humanoids](https://mlsanity.com/paper/2609.15988/) (2026) - [DODGER: Safety-Guided Reinforcement Learning for Robot Navigation Among Dynamic Obstacles](https://mlsanity.com/paper/2609.38873/) (2026) - [HULK: Learning Whole-Body Forceful Loco-Manipulation for Humanoids](https://mlsanity.com/paper/2610.08970/) (2026) - [FAITH: Feasibility-Aware Safety-Filtered RL for High-Dimensional Systems](https://mlsanity.com/paper/2610.12432/) (2026) - [CSF: Contextual Safety Filtering for Motion Generators](https://mlsanity.com/paper/2610.12467/) (2026) - [Beyond Safe Execution: Safe Online Reinforcement Learning for Autonomous Driving with Value Calibration and Adaptive Guidance](https://mlsanity.com/paper/2610.17799/) (2026) - [Safe Reinforcement Learning with Receding-Horizon Safety Critic and Differentiable Action Shields](https://mlsanity.com/paper/2610.20255/) (2026) ## Abstract (arXiv) Reinforcement learning (RL), while powerful and expressive, can often prioritize performance at the expense of safety. Yet safety violations can lead to catastrophic outcomes in real-world deployments. Control Barrier Functions (CBFs) offer a principled method to enforce dynamic safety -- traditionally deployed online via safety filters. While the result is safe behavior, the fact that the RL policy does not have knowledge of the CBF can lead to conservative behaviors. This paper proposes CBF-RL, a framework for generating safe behaviors with RL by enforcing CBFs in training. CBF-RL has two key attributes: (1) minimally modifying a nominal RL policy to encode safety constraints via a CBF term, (2) and safety filtering of the policy rollouts in training. Theoretically, we prove that continuous-time safety filters can be deployed via closed-form expressions on discrete-time roll-outs. Practically, we demonstrate that CBF-RL internalizes the safety constraints in the learned policy -- both enforcing safer actions and biasing towards safer rewards -- enabling safe deployment without the need for an online safety filter. We validate our framework through ablation studies on navigation tasks and on the Unitree G1 humanoid robot, where CBF-RL enables safer exploration, faster convergence, and robust performance under uncertainty, enabling the humanoid robot to avoid obstacles and climb stairs safely in real-world settings without a runtime safety filter.