mlsanityData to 7 Oct 2026

Provably Safe Reinforcement Learning: Conceptual Analysis, Survey, and Benchmarking

arXiv 2205.06750 ↗First posted 13 May 2022cs.LGcited by 31

Not on the map. Why?

Abstract (arXiv)

Ensuring the safety of reinforcement learning (RL) algorithms is crucial to unlock their potential for many real-world tasks. However, vanilla RL and most safe RL approaches do not guarantee safety. In recent years, several methods have been proposed to provide hard safety guarantees for RL, which is essential for applications where unsafe actions could have disastrous consequences. Nevertheless, there is no comprehensive comparison of these provably safe RL methods. Therefore, we introduce a categorization of existing provably safe RL methods, present the conceptual foundations for both continuous and discrete action spaces, and empirically benchmark existing methods. We categorize the methods based on how they adapt the action: action replacement, action projection, and action masking. Our experiments on an inverted pendulum and a quadrotor stabilization task indicate that action replacement is the best-performing approach for these applications despite its comparatively simple realization. Furthermore, adding a reward penalty, every time the safety verification is engaged, improved training performance in our experiments. Finally, we provide practical guidance on selecting provably safe RL approaches depending on the safety specification, RL algorithm, and type of action space.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 56 of its 119 references that are in the map. First posted on arXiv 13 May 2022.

Builds on

56 of 119 references.

References · 56

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2017Deep reinforcement learning from human preferencescited by 3,846
  4. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  5. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  6. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  7. 2017Constrained Policy Optimizationcited by 928
  8. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
  9. 2017Safe Reinforcement Learning via Shieldingcited by 427
  10. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 388
  11. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 347
  12. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
38 more
  1. 2019Control Barrier Functions: Theory and Applicationscited by 332
  2. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 302
  3. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 278
  4. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 256
  5. 2018Safe Exploration in Continuous Action Spacescited by 237
  6. 2020Projection-Based Constrained Policy Optimizationcited by 195
  7. 2018Verifiable Reinforcement Learning via Policy Extractioncited by 176
  8. 2020A Closer Look at Invalid Action Masking in Policy Gradient Algorithmscited by 160
  9. 2020Recovery RL: Safe Reinforcement Learning with Learned Recovery Zonescited by 156
  10. 2017A General Safety Framework for Learning-Based Control in Uncertain Robotic Systemscited by 150
  11. 2018Assessing Generalization in Deep Reinforcement Learningcited by 132
  12. 2016Reinforcement Learning With Temporal Logic Rewardscited by 114
  13. 2018A predictive safety filter for learning-based control of constrained nonlinear dynamical systemscited by 91
  14. 2017Online algorithms for POMDPs with continuous state, action, and observation spacescited by 82
  15. 2019Reinforcement Learning for Temporal Logic Control Synthesis with Probabilistic Satisfaction Guaranteescited by 77
  16. 2018Omega-Regular Objectives in Model-Free Reinforcement Learningcited by 74
  17. 2016Safe Exploration in Finite Markov Decision Processes with Gaussian Processescited by 72
  18. 2019Safe Reinforcement Learning Using Robust MPCcited by 72
  19. 2018Foundations for Restraining Bolts: Reinforcement Learning with LTLf/LDLf restraining specificationscited by 69
  20. 2019Learning for Safety-Critical Control with Control Barrier Functionscited by 66
  21. 2016Q-Learning for Robust Satisfaction of Signal Temporal Logic Specificationscited by 65
  22. 2017OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real Worldcited by 64
  23. 2020Neurosymbolic Reinforcement Learning with Formally Verified Explorationcited by 49
  24. 2020Cautious Reinforcement Learning with Logical Constraintscited by 44
  25. 2021Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logiccited by 38
  26. 2020Verifiably Safe Exploration for End-to-End Reinforcement Learningcited by 32
  27. 2020Safe Reinforcement Learning via Projection on a Safe Set: How to Achieve Optimality?cited by 29
  28. 2020Reachability-based Trajectory Safeguard (RTS): A Safe and Fast Reinforcement Learning Safety Layer for Continuous Controlcited by 28
  29. 2020On Robustness Metrics for Learning STL Taskscited by 27
  30. 2022Safe Reinforcement Learning Using Black-Box Reachability Analysiscited by 22
  31. 2021On the (In)Tractability of Reinforcement Learning for LTL Objectivescited by 18
  32. 2021Computationally Efficient Safe Reinforcement Learning for Power Systemscited by 17
  33. 2019Verifiably Safe Off-Model Reinforcement Learningcited by 16
  34. 2022Provably Safe Deep Reinforcement Learning for Robotic Manipulation in Human Environmentscited by 16
  35. 2019Autonomous Driving using Safe Reinforcement Learning by Incorporating a Regret-based Human Lane-Changing Decision Modelcited by 14
  36. 2020Online Shielding for Stochastic Systemscited by 11
  37. 2021Scalable Learning of Safety Guarantees for Autonomous Systems using Hamilton-Jacobi Reachabilitycited by 10
  38. 2021A Survey of Deep Reinforcement Learning Algorithms for Motion Planning and Control of Autonomous Vehiclescited by 10

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 11; 2025: 15; 2026: 5.

31 papers.

Sort:

By area:

  1. 2024Excluding the Irrelevant: Focusing Reinforcement Learning through Continuous Action Maskingcited by 17
  2. 2024Dynamic Model Predictive Shielding for Provably Safe Reinforcement Learningcited by 11
  3. 2024Safety Filtering While Training: Improving the Performance and Sample Efficiency of Reinforcement Learning Agentscited by 11
  4. 2025CBF-RL: Safety Filtering Reinforcement Learning in Training with Control Barrier Functionscited by 8
  5. 2024Compositional Shielding and Reinforcement Learning for Multi-Agent Systemscited by 7
  6. 2025Safe Reinforcement Learning using Action Projection: Safeguard the Policy or the Environment?cited by 5
  7. 2025From Demonstrations to Safe Deployment: Path-Consistent Safety Filtering for Diffusion Policiescited by 4
  8. 2024CommonPower: A Framework for Safe Data-Driven Smart Grid Controlcited by 3
  9. 2024Safety-Driven Deep Reinforcement Learning Framework for Cobots: A Sim2Real Approachcited by 3
  10. 2024Provable Traffic Rule Compliance in Safe Reinforcement Learning on the Open Seacited by 2
  11. 2025CAMEL: Continuous Action Masking Enabled by Large Language Models for Reinforcement Learningcited by 2
  12. 2025A Provable Approach for End-to-End Safe Reinforcement Learningcited by 2
19 more
  1. 2025Safe and Compliant Cross-Market Trade Execution via Constrained RL and Zero-Knowledge Auditscited by 2
  2. 2025Improving Stochastic Action-Constrained Reinforcement Learning via Truncated Distributionscited by 2
  3. 2026Shields to Guarantee Probabilistic Safety in MDPscited by 2
  4. 2026Any-Body Guard: Universal Safeguarding for Manipulation Policies via Action Maskingcited by 2
  5. 2024Numeric Reward Machinescited by 1
  6. 2024Can an AI Agent Safely Run a Government? Existence of Probably Approximately Aligned Policiescited by 1
  7. 2025Safe Learning Under Irreversible Dynamics via Asking for Helpcited by 1
  8. 2025Leveraging Analytic Gradients in Provably Safe Reinforcement Learningcited by 1
  9. 2025Predictive Safety Shield for Dyna-Q Reinforcement Learningcited by 1
  10. 2026Safe Reinforcement Learning via Recovery-based Shielding with Gaussian Process Dynamics Modelscited by 1
  11. 2024Heuristic Algorithm-based Action Masking Reinforcement Learning (HAAM-RL) with Ensemble Inference Methodcited by 0
  12. 2024Training Verifiably Robust Agents Using Set-Based Reinforcement Learningcited by 0
  13. 2025Falsification-driven reinforcement learning for maritime motion planningcited by 0
  14. 2025Learning When Not to Learn: Risk-Sensitive Abstention in Bandits with Unbounded Rewardscited by 0
  15. 2025Reinforcement Learning for Long-Horizon Unordered Tasks: From Boolean to Coupled Reward Machinescited by 0
  16. 2025LEARN: Learning End-to-End Aerial Resource-Constrained Multi-Robot Navigationcited by 0
  17. 2025Shielded Controller Units for RL with Operational Constraints Applied to Remote Microgridscited by 0
  18. 2026Safe Online Learning via Smooth Safety-Structured Policy Compositioncited by 0
  19. 2026Decentralized Safe Multi-Agent Reinforcement Learning via Predictive Shieldingcited by 0