mlsanityData to 7 Oct 2026

Safety Filtering While Training: Improving the Performance and Sample Efficiency of Reinforcement Learning Agents

arXiv 2410.11671 ↗First posted 15 Oct 2024cs.RO cs.LG cs.SYcited by 11

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning (RL) controllers are flexible and performant but rarely guarantee safety. Safety filters impart hard safety guarantees to RL controllers while maintaining flexibility. However, safety filters can cause undesired behaviours due to the separation between the controller and the safety filter, often degrading performance and robustness. In this paper, we analyze several modifications to incorporating the safety filter in training RL controllers rather than solely applying it during evaluation. The modifications allow the RL controller to learn to account for the safety filter, improving performance. This paper presents a comprehensive analysis of training RL with safety filters, featuring simulated and real-world experiments with a Crazyflie 2.0 drone. We examine how various training modifications and hyperparameters impact performance, sample efficiency, safety, and chattering. Our findings serve as a guide for practitioners and researchers focused on safety filters and safe RL.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 21 of its 37 references that are in the map. First posted on arXiv 15 Oct 2024.

Builds on

21 of 37 references.

References · 21

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2017Constrained Policy Optimizationcited by 928
  3. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
  4. 2019Control Barrier Functions: Theory and Applicationscited by 332
  5. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 256
  6. 2018Safe Exploration in Continuous Action Spacescited by 237
  7. 2018A predictive safety filter for learning-based control of constrained nonlinear dynamical systemscited by 91
  8. 2023The Safety Filter: A Unified View of Safety-Critical Control in Autonomous Systemscited by 77
  9. 2017OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real Worldcited by 64
  10. 2019acados: a modular open-source framework for fast embedded optimal controlcited by 61
  11. 2021safe-control-gym: a Unified Benchmark Suite for Safe Learning-based Control and Reinforcement Learning in Roboticscited by 42
  12. 2022Provably Safe Reinforcement Learning: Conceptual Analysis, Survey, and Benchmarkingcited by 31
9 more
  1. 2022Provably Safe Reinforcement Learning via Action Projection using Reachability Analysis and Polynomial Zonotopescited by 23
  2. 2021Safe Reinforcement Learning with Chance-constrained Model Predictive Controlcited by 18
  3. 2019A computationally efficient robust model predictive control framework for uncertain nonlinear systems -- extended versioncited by 12
  4. 2020Zeus: A System Description of the Two-Time Winner of the Collegiate SAE AutoDrive Competitioncited by 8
  5. 2022Differentiable Safe Controller Design through Control Barrier Functionscited by 8
  6. 2016Design of a Trajectory Tracking Controller for a Nanoquadcoptercited by 4
  7. 2023Learn With Imagination: Safe Set Guided State-wise Constrained Policy Optimizationcited by 4
  8. 2023Multi-Step Model Predictive Safety Filters: Reducing Chattering by Increasing the Prediction Horizoncited by 3
  9. 2023A Remote Sim2real Aerial Competition: Fostering Reproducibility and Solutions' Diversity in Robotics Challengescited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 6.

11 papers.

  1. 2025CBF-RL: Safety Filtering Reinforcement Learning in Training with Control Barrier Functionscited by 8
  2. 2025Safe Reinforcement Learning using Action Projection: Safeguard the Policy or the Environment?cited by 5
  3. 2026Hard Constraints Meet Soft Generation: Guaranteed Feasibility for LLM-based Combinatorial Optimizationcited by 3
  4. 2025Provably Optimal Reinforcement Learning under Safety Filteringcited by 2
  5. 2026Any-Body Guard: Universal Safeguarding for Manipulation Policies via Action Maskingcited by 2
  6. 2025Leveraging Analytic Gradients in Provably Safe Reinforcement Learningcited by 1
  7. 2026Dyna-Style Safety Augmented Reinforcement Learning: Staying Safe in the Face of Uncertaintycited by 1
  8. 2026Uncertainty-Aware Predictive Safety Filters for Probabilistic Neural Network Dynamicscited by 1
  9. 2026Permissive Safety Through Trusted Inference: Verifiable Belief-Space Neural Safety Filters for Assured Interactive Roboticscited by 1
  10. 2025LEARN: Learning End-to-End Aerial Resource-Constrained Multi-Robot Navigationcited by 0
  11. 2026All Models are Wrong, Knowing Where is Useful: On Model Uncertainty in Reinforcement Learningcited by 0