mlsanityData to 14 Oct 2026

State-wise Safe Reinforcement Learning: A Survey

arXiv 2302.03122 ↗First posted 6 Feb 2023cs.LG cs.AI cs.ROcited by 55

Not on the map. Why?

Cited by 33 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Despite the tremendous success of Reinforcement Learning (RL) algorithms in simulation environments, applying RL to real-world applications still faces many challenges. A major concern is safety, in another word, constraint satisfaction. State-wise constraints are one of the most common constraints in real-world applications and one of the most challenging constraints in Safe RL. Enforcing state-wise constraints is necessary and essential to many challenging tasks such as autonomous driving, robot manipulation. This paper provides a comprehensive review of existing approaches that address state-wise constraints in RL. Under the framework of State-wise Constrained Markov Decision Process (SCMDP), we will discuss the connections, differences, and trade-offs of existing approaches in terms of (i) safety guarantee and scalability, (ii) safety and reward performance, and (iii) safety after convergence and during training. We also summarize limitations of current methods and discuss potential future directions.

Key facts. Cited by 55 papers in the map (data through 14 Oct 2026). Builds on 33 of its 63 references that are in the map. First posted on arXiv 6 Feb 2023.

Builds on

33 of 63 references.

References · 33

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Constrained Policy Optimizationcited by 983
  3. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 371
  4. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 366
  5. 2017On Kernelized Multi-armed Banditscited by 352
  6. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 297
  7. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 289
  8. 2018Safe Exploration in Continuous Action Spacescited by 247
  9. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 181
  10. 2020Recovery RL: Safe Reinforcement Learning with Learned Recovery Zonescited by 175
  11. 2017A General Safety Framework for Learning-Based Control in Uncertain Robotic Systemscited by 163
  12. 2019Lyapunov-based Safe Policy Optimization for Continuous Controlcited by 145
21 more
  1. 2020Safe Reinforcement Learning in Constrained Markov Decision Processescited by 115
  2. 2020Conservative Safety Critics for Explorationcited by 113
  3. 2018Accelerated Primal-Dual Policy Optimization for Safe Reinforcement Learningcited by 62
  4. 2019Value constrained model-free continuous controlcited by 34
  5. 2020Reachability-based Trajectory Safeguard (RTS): A Safe and Fast Reinforcement Learning Safety Layer for Continuous Controlcited by 28
  6. 2021Joint Synthesis of Safety Certificate and Safe Control Policy using Constrained Reinforcement Learningcited by 27
  7. 2021Safe Control with Neural Network Dynamic Modelscited by 22
  8. 2022Evaluating Model-free Reinforcement Learning toward Safety-critical Taskscited by 22
  9. 2019Safe Control Algorithms Using Energy Functions: A Unified Framework, Benchmark, and New Directionscited by 17
  10. 2020ShieldNN: A Provably Safe NN Filter for Unsafe NN Controllerscited by 17
  11. 2023AutoCost: Evolving Intrinsic Cost for Zero-violation Reinforcement Learningcited by 17
  12. 2022Probabilistic Safeguard for Reinforcement Learning Using Safety Index Guided Gaussian Process Modelscited by 15
  13. 2022Reinforcement Learning with Automated Auxiliary Loss Searchcited by 13
  14. 2022Persistently Feasible Robust Safe Control by Safety Index Synthesis and Convex Semi-Infinite Programmingcited by 12
  15. 2022Safety Index Synthesis via Sum-of-Squares Programmingcited by 12
  16. 2019Safe Adaptation with Multiplicative Uncertainties Using Robust Safe Set Algorithmcited by 8
  17. 2019Stochastic Variance Reduction for Deep Q-learningcited by 7
  18. 2020Contact-Rich Trajectory Generation in Confined Environments Using Iterative Convex Optimizationcited by 7
  19. 2021Provably Safe Tolerance Estimation for Robot Arms via Sum-of-Squares Programmingcited by 6
  20. 2021Learn Zero-Constraint-Violation Policy in Model-Free Constrained Reinforcement Learningcited by 4
  21. 2019How to Evaluate Proving Grounds for Self-Driving? A Quantitative Approachcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 8; 2024: 25; 2025: 12; 2026: 10.

55 papers.

Sort:

By area:

  1. 2024Agile But Safe: Learning Collision-Free High-Speed Legged Locomotioncited by 73
  2. 2024Safe Offline Reinforcement Learning with Feasibility-Guided Diffusion Modelcited by 64
  3. 2024A Survey of Constraint Formulations in Safe Reinforcement Learningcited by 44
  4. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Surveycited by 32
  5. 2024Hamilton-Jacobi Reachability in Reinforcement Learning: A Surveycited by 16
  6. 2024Dynamic Model Predictive Shielding for Provably Safe Reinforcement Learningcited by 15
  7. 2025A Survey on Imitation Learning for Contact-Rich Tasks in Roboticscited by 14
  8. 2023GUARD: A Safe Reinforcement Learning Benchmarkcited by 13
  9. 2024Verification of Neural Control Barrier Functions with Symbolic Derivative Bounds Propagationcited by 12
  10. 2024A Review of Safe Reinforcement Learning Methods for Modern Power Systemscited by 11
  11. 2023State-wise Constrained Policy Optimizationcited by 9
  12. 2023Safe Deep Policy Adaptationcited by 9
38 more
  1. 2024Enhancing Efficiency of Safe Reinforcement Learning via Sample Manipulationcited by 9
  2. 2023Multi-Agent Reinforcement Learning Guided by Signal Temporal Logic Specificationscited by 7
  3. 2024Safe Multi-Agent Reinforcement Learning with Convergence to Generalized Nash Equilibriumcited by 6
  4. 2023Safe Reinforcement Learning with Dead-Ends Avoidance and Recoverycited by 5
  5. 2024Deterministic Policies for Constrained Reinforcement Learning in Polynomial Timecited by 5
  6. 2024Action Mapping for Reinforcement Learning in Continuous Environments with Constraintscited by 5
  7. 2025Emergent Risk Awareness in Rational Agents under Resource Constraintscited by 5
  8. 2023Robust Safe Reinforcement Learning under Adversarial Disturbancescited by 4
  9. 2023Learning Predictive Safety Filter via Decomposition of Robust Invariant Setcited by 3
  10. 2024Implicit Safe Set Algorithm for Provably Safe Reinforcement Learningcited by 3
  11. 2026Speedup Patch: Learning a Plug-and-Play Policy to Accelerate Embodied Manipulationcited by 3
  12. 2023Absolute Policy Optimizationcited by 2
  13. 2024Meta-Control: Automatic Model-based Control Synthesis for Heterogeneous Robot Skillscited by 2
  14. 2024Combining Deep Reinforcement Learning with a Jerk-Bounded Trajectory Generator for Kinematically Constrained Motion Planningcited by 2
  15. 2025Leveraging Constraint Violation Signals For Action-Constrained Reinforcement Learningcited by 2
  16. 2025A Survey of Reinforcement Learning-Based Motion Planning for Autonomous Driving: Lessons Learned from a Driving Task Perspectivecited by 2
  17. 2025Learning to Undo: Rollback-Augmented Reinforcement Learning with Reversibility Signalscited by 2
  18. 2026EmbodiedGovBench: A Benchmark for Governance, Recovery, and Upgrade Safety in Embodied Agent Systemscited by 2
  19. 2024Uniformly Safe RL with Objective Suppression for Multi-Constraint Safety-Critical Applicationscited by 1
  20. 2024SECRM-2D: RL-Based Efficient and Comfortable Route-Following Autonomous Driving with Analytic Safety Guaranteescited by 1
  21. 2024Safe Reinforcement Learning Filter for Multicopter Collision-Free Tracking under disturbancescited by 1
  22. 2024Anytime-Constrained Equilibria in Polynomial Timecited by 1
  23. 2025Polynomial-Time Approximability of Constrained Reinforcement Learningcited by 1
  24. 2025Safe Learning Under Irreversible Dynamics via Asking for Helpcited by 1
  25. 2026Dyna-Style Safety Augmented Reinforcement Learning: Staying Safe in the Face of Uncertaintycited by 1
  26. 2026Provably Safe, Yet Scalable Reinforcement Learningcited by 1
  27. 2026Safe Execution of RL Policies Via Acceleration-Based CBF-QP Constraint Enforcement for Real-World Robotic Deploymentscited by 1
  28. 2024Make Safe Decisions in Power System: Safe Reinforcement Learning Based Pre-decision Making for Voltage Stability Emergency Controlcited by 0
  29. 2024GenSafe: A Generalizable Safety Enhancer for Safe Reinforcement Learning Algorithms Based on Reduced Order Markov Decision Process Modelcited by 0
  30. 2024A Safe Exploration Strategy for Model-free Task Adaptation in Safety-constrained Grid Environmentscited by 0
  31. 2024Revisiting Safe Exploration in Safe Reinforcement learningcited by 0
  32. 2024Absolute State-wise Constrained Policy Optimization: High-Probability State-wise Constraints Satisfactioncited by 0
  33. 2024A Safety Modulator Actor-Critic Method in Model-Free Safe Reinforcement Learning and Application in UAV Hoveringcited by 0
  34. 2025A Graph-Based Reinforcement Learning Approach with Frontier Potential Based Reward for Safe Cluttered Environment Explorationcited by 0
  35. 2025Continuous World Coverage Path Planning for Fixed-Wing UAVs using Deep Reinforcement Learningcited by 0
  36. 2025Hierarchical Reinforcement Learning with Low-Level MPC for Multi-Agent Controlcited by 0
  37. 2025Tail-Safe Hedging: Explainable Risk-Sensitive Reinforcement Learning with a White-Box CBF--QP Safety Layer in Arbitrage-Free Marketscited by 0
  38. 2025Platform-Agnostic Reinforcement Learning Framework for Safe Exploration of Cluttered Environments with Graph Attentioncited by 0

50 most cited of 55 All 55 (JSON)