mlsanityData to 14 Oct 2026

Back to Base: Towards Hands-Off Learning via Safe Resets with Reach-Avoid Safety Filters

arXiv 2501.02620 ↗First posted 5 Jan 2025eess.SY cs.RO cs.SYcited by 2

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

Designing controllers that accomplish tasks while guaranteeing safety constraints remains a significant challenge. We often want an agent to perform well in a nominal task, such as environment exploration, while ensuring it can avoid unsafe states and return to a desired target by a specific time. In particular we are motivated by the setting of safe, efficient, hands-off training for reinforcement learning in the real world. By enabling a robot to safely and autonomously reset to a desired region (e.g., charging stations) without human intervention, we can enhance efficiency and facilitate training. Safety filters, such as those based on control barrier functions, decouple safety from nominal control objectives and rigorously guarantee safety. Despite their success, constructing these functions for general nonlinear systems with control constraints and system uncertainties remains an open problem. This paper introduces a safety filter obtained from the value function associated with the reach-avoid problem. The proposed safety filter minimally modifies the nominal controller while avoiding unsafe regions and guiding the system back to the desired target set. By preserving policy performance while allowing safe resetting, we enable efficient hands-off reinforcement learning and advance the feasibility of safe training for real world robots. We demonstrate our approach using a modified version of soft actor-critic to safely train a swing-up task on a modified cartpole stabilization problem.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 24 of its 31 references that are in the map. First posted on arXiv 5 Jan 2025.

Builds on

24 of 31 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Deep Reinforcement Learning for Reach-Avoid-Stay Problems3 Oct

Other references · 23

  1. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  2. 2020dm_control: Software and Tasks for Continuous Controlcited by 375
  3. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 371
  4. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 366
  5. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 289
  6. 2016Control Barrier Function Based Quadratic Programs for Safety Critical Systemscited by 287
  7. 2017Hamilton-Jacobi Reachability: A Brief Overview and Recent Advancescited by 201
  8. 2020Recovery RL: Safe Reinforcement Learning with Learned Recovery Zonescited by 175
  9. 2020The Ingredients of Real-World Robotic Reinforcement Learningcited by 117
  10. 2020DeepReach: A Deep Learning Approach to High-Dimensional Reachabilitycited by 91
  11. 2020Reinforcement Learning for Safety-Critical Control under Model Uncertainty, using Control Lyapunov Functions and Control Barrier Functionscited by 68
  12. 2021Safe Reinforcement Learning Using Robust Control Barrier Functionscited by 59
11 more
  1. 2022Reachability Constrained Reinforcement Learningcited by 59
  2. 2021Robust Control Barrier-Value Functions for Safety-Critical Controlcited by 51
  3. 2021Safety and Liveness Guarantees through Reach-Avoid Reinforcement Learningcited by 49
  4. 2014Reach-Avoid Problems with Time-Varying Dynamics, Targets and Constraintscited by 44
  5. 2023How to Train Your Neural Control Barrier Function: Learning Safety Filters for Complex Input-Constrained Systemscited by 44
  6. 2020Multi-Layered Safety for Legged Robots via Control Barrier Functions and Model Predictive Controlcited by 25
  7. 2018Input-to-State Safety With Control Barrier Functionscited by 15
  8. 2021Backup Control Barrier Functions: Formulation and Comparative Studycited by 12
  9. 2022Safe and Efficient Reinforcement Learning Using Disturbance-Observer-Based Control Barrier Functionscited by 11
  10. 2023Feasibility and Regularity Analysis of Safe Stabilizing Controllers under Uncertaintycited by 4
  11. 2024Guaranteed Reach-Avoid for Black-Box Systems through Narrow Gaps via Neural Network Reachabilitycited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2026Synthesis and Deployment of Maximal Robust Control Barrier Functions through Adversarial Reinforcement Learningcited by 5
  2. 2025Probabilistic Safety Guarantee for Stochastic Control Systems Using Average Reward MDPscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Turning Safety into Competence: Minimally Exploitable Robot Policies via Safety-Filtered Reinforcement Learning
  2. 2024A Dynamic Safety Shield for Safe and Efficient Reinforcement Learning of Navigation Tasks
  3. 2025Provably Optimal Reinforcement Learning under Safety Filtering
  4. 2022Safe Inverse Reinforcement Learning via Control Barrier Function
  5. 2026CAPSULE: Control-Theoretic Action Perturbations for Safe Uncertainty-Aware Reinforcement Learning
  6. 2025V-OCBF: Learning Safety Filters from Offline Data via Value-Guided Offline Control Barrier Functions
  7. 2024Handling Long-Term Safety and Uncertainty in Safe Reinforcement Learning
  8. 2025Passivity-Centric Safe Reinforcement Learning for Contact-Rich Robotic Tasks