mlsanityData to 7 Oct 2026

Safe Reinforcement Learning via Confidence-Based Filters

arXiv 2207.01337 ↗First posted 4 Jul 2022cs.LG cs.AI cs.SYcited by 5

Not on the map. Why?

Abstract (arXiv)

Ensuring safety is a crucial challenge when deploying reinforcement learning (RL) to real-world systems. We develop confidence-based safety filters, a control-theoretic approach for certifying state safety constraints for nominal policies learned via standard RL techniques, based on probabilistic dynamics models. Our approach is based on a reformulation of state constraints in terms of cost functions, reducing safety verification to a standard RL task. By exploiting the concept of hallucinating inputs, we extend this formulation to determine a "backup" policy that is safe for the unknown system with high probability. Finally, the nominal policy is minimally adjusted at every time step during a roll-out towards the backup policy, such that safe recovery can be guaranteed afterwards. We provide formal safety guarantees, and empirically demonstrate the effectiveness of our approach.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 17 of its 29 references that are in the map. First posted on arXiv 4 Jul 2022.

Builds on

17 of 29 references.

References · 17

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2017Constrained Policy Optimizationcited by 928
  4. 2017Robust Adversarial Reinforcement Learningcited by 439
  5. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
  6. 2019Challenges of Real-World Reinforcement Learningcited by 282
  7. 2017A General Safety Framework for Learning-Based Control in Uncertain Robotic Systemscited by 150
  8. 2019Constrained Reinforcement Learning Has Zero Duality Gapcited by 144
  9. 2020Provably Efficient Safe Exploration via Primal-Dual Policy Optimizationcited by 131
  10. 2018Learning-based Model Predictive Control for Safe Explorationcited by 116
  11. 2020Efficient Model-Based Reinforcement Learning through Optimistic Policy Search and Planningcited by 83
  12. 2019Learning for Safety-Critical Control with Control Barrier Functionscited by 66
5 more
  1. 2016Learning Model Predictive Control for iterative tasks. A Data-Driven Control Frameworkcited by 38
  2. 2018Safely Learning to Control the Constrained Linear Quadratic Regulatorcited by 31
  3. 2019Probabilistic model predictive safety certification for learning-based controlcited by 20
  4. 2021Combining Pessimism with Optimism for Robust and Efficient Model-Based Deep Reinforcement Learningcited by 14
  5. 2020Data selection for multi-task learning under dynamic constraintscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 2.

5 papers.

  1. 2024ActSafe: Active Exploration with Safety Constraints for Reinforcement Learningcited by 22
  2. 2025Koopman-Equivariant Gaussian Processescited by 6
  3. 2026Safe Exploration via Policy Priorscited by 6
  4. 2025Distributed Risk-Sensitive Safety Filters for Uncertain Discrete-Time Systemscited by 0
  5. 2026Sampling-Based Safe Reinforcement Learningcited by 0