mlsanityData to 7 Oct 2026

Enforcing Hard Constraints with Soft Barriers: Safe Reinforcement Learning in Unknown Stochastic Environments

arXiv 2209.15090 ↗First posted 29 Sep 2022eess.SY cs.LG cs.SYcited by 54

Not on the map. Why?

Abstract (arXiv)

It is quite challenging to ensure the safety of reinforcement learning (RL) agents in an unknown and stochastic environment under hard constraints that require the system state not to reach certain specified unsafe regions. Many popular safe RL methods such as those based on the Constrained Markov Decision Process (CMDP) paradigm formulate safety violations in a cost function and try to constrain the expectation of cumulative cost under a threshold. However, it is often difficult to effectively capture and enforce hard reachability-based safety constraints indirectly with such constraints on safety violation costs. In this work, we leverage the notion of barrier function to explicitly encode the hard safety constraints, and given that the environment is unknown, relax them to our design of generative-model-based soft barrier functions. Based on such soft barriers, we propose a safe RL approach that can jointly learn the environment and optimize the control policy, while effectively avoiding unsafe regions with safety probability optimization. Experiments on a set of examples demonstrate that our approach can effectively enforce hard safety constraints and significantly outperform CMDP-based baseline methods in system safe rate measured via simulations.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). Builds on 47 of its 75 references that are in the map. First posted on arXiv 29 Sep 2022.

Builds on

47 of 75 references.

References · 47

  1. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 388
  2. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 347
  3. 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 320
  4. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 302
  5. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 278
  6. 2020Scalable Gradients for Stochastic Differential Equationscited by 264
  7. 2016Control Barrier Function Based Quadratic Programs for Safety Critical Systemscited by 261
  8. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 256
  9. 2018Safe Exploration in Continuous Action Spacescited by 237
  10. 2020FLAMBE: Structural Complexity and Representation Learning of Low Rank MDPscited by 232
  11. 2021Reinforcement learning based recommender systems: A surveycited by 187
  12. 2020Neural Lyapunov Controlcited by 142
35 more
  1. 2020Provably Efficient Safe Exploration via Primal-Dual Policy Optimizationcited by 131
  2. 2020CRPO: A New Approach for Safe Reinforcement Learning with Convergence Guaranteecited by 128
  3. 2019Model-Based Reinforcement Learning with a Generative Model is Minimax Optimalcited by 124
  4. 2020Breaking the Sample Size Barrier in Model-Based Reinforcement Learning with a Generative Modelcited by 108
  5. 2020Conservative Safety Critics for Explorationcited by 107
  6. 2021Safe Nonlinear Control Using Robust Neural Lyapunov-Barrier Functionscited by 91
  7. 2016Safe Exploration in Finite Markov Decision Processes with Gaussian Processescited by 72
  8. 2021Learning Safe Multi-Agent Control with Decentralized Neural Barrier Certificatescited by 71
  9. 2019Learning for Safety-Critical Control with Control Barrier Functionscited by 66
  10. 2020Reinforcement Learning for Safety-Critical Control under Model Uncertainty, using Control Lyapunov Functions and Control Barrier Functionscited by 61
  11. 2022Penalized Proximal Policy Optimization for Safe Reinforcement Learningcited by 60
  12. 2022Constrained Policy Optimization via Bayesian World Modelscited by 55
  13. 2022Reachability Constrained Reinforcement Learningcited by 52
  14. 2021Achieving Zero Constraint Violation for Constrained Reinforcement Learning via Primal-Dual Approachcited by 46
  15. 2019Robust Model Predictive Shielding for Safe Reinforcement Learning with Stochastic Dynamicscited by 45
  16. 2021Safe Reinforcement Learning Using Advantage-Based Interventioncited by 45
  17. 2020ContactNets: Learning Discontinuous Contact Dynamics with Smooth, Implicit Representationscited by 38
  18. 2021POLAR: A Polynomial Arithmetic Framework for Verifying Neural-Network Controlled Systemscited by 38
  19. 2021Learning Barrier Certificates: Towards Safe Reinforcement Learning with Zero Training-time Violationscited by 36
  20. 2020Learning with Safety Constraints: Sample Complexity of Reinforcement Learning for Constrained MDPscited by 32
  21. 2021Model-based Constrained Reinforcement Learning using Generalized Control Barrier Functioncited by 30
  22. 2020Reachability-based Trajectory Safeguard (RTS): A Safe and Fast Reinforcement Learning Safety Layer for Continuous Controlcited by 28
  23. 2021Safe Pontryagin Differentiable Programmingcited by 25
  24. 2020Learning Hybrid Control Barrier Functions from Datacited by 24
  25. 2022Joint Differentiable Optimization and Verification for Certified Reinforcement Learningcited by 11
  26. 2020Sequential Transfer in Reinforcement Learning with a Generative Modelcited by 10
  27. 2021Continuous Latent Process Flowscited by 10
  28. 2021Fundamental Challenges in Deep Learning for Stiff Contact Dynamicscited by 9
  29. 2022Physics-Aware Safety-Assured Design of Hierarchical Neural Network based Plannercited by 9
  30. 2020Energy-Efficient Control Adaptation with Safety Guarantees for Learning-Enabled Cyber-Physical Systemscited by 7
  31. 2022Safety-driven Interactive Planning for Neural Network-based Lane Changingcited by 6
  32. 2020Opportunistic Intermittent Control with Safety Guarantees for Autonomous Systemscited by 5
  33. 2021Learning-based Framework for Sensor Fault-Tolerant Building HVAC Control with Model-assisted Learningcited by 4
  34. 2021Cocktail: Learn a Better Neural Network Controller from Multiple Experts via Adaptive Mixing and Robust Distillationcited by 3
  35. 2020Cross-Layer Design of Automotive Systemscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 17; 2025: 14; 2026: 14.

54 papers.

Sort:

By area:

  1. 2024Safe Offline Reinforcement Learning with Feasibility-Guided Diffusion Modelcited by 62
  2. 2024A Survey of Constraint Formulations in Safe Reinforcement Learningcited by 42
  3. 2023Empowering Autonomous Driving with Large Language Models: A Safety Perspectivecited by 34
  4. 2023Iterative Reachability Estimation for Safe Reinforcement Learningcited by 33
  5. 2025VLSA: Vision-Language-Action Models with Plug-and-Play Safety Constraint Layercited by 32
  6. 2023Safe Exploration in Reinforcement Learning: A Generalized Formulation and Algorithmscited by 27
  7. 2024Can We Trust Embodied Agents? Exploring Backdoor Attacks against Embodied LLM-based Decision-Making Systemscited by 23
  8. 2023POLAR-Express: Efficient and Precise Formal Reachability Analysis of Neural-Network Controlled Systemscited by 12
  9. 2023Compositional Policy Learning in Stochastic Control Systems with Formal Guaranteescited by 12
  10. 2024Safe Reinforcement Learning on the Constraint Manifold: Theory and Applicationscited by 12
  11. 2023Provably Safe Reinforcement Learning with Step-wise Violation Constraintscited by 10
  12. 2024Boosting Reinforcement Learning with Strongly Delayed Feedback Through Auxiliary Short Delayscited by 10
38 more
  1. 2023State-Wise Safe Reinforcement Learning With Pixel Observationscited by 9
  2. 2024Variational Delayed Policy Optimizationcited by 9
  3. 2024Verified Safe Reinforcement Learning for Neural Network Dynamic Modelscited by 7
  4. 2023Reduced Policy Optimization for Continuous Control with Hard Constraintscited by 5
  5. 2024Deterministic Policies for Constrained Reinforcement Learning in Polynomial Timecited by 5
  6. 2024Enhancing Safety in Reinforcement Learning with Human Feedback via Rectified Policy Optimizationcited by 5
  7. 2024Act as You Learn: Adaptive Decision-Making in Non-Stationary Markov Decision Processescited by 4
  8. 2025Hierarchical Multi-Agent Reinforcement Learning with Control Barrier Functions for Safety-Critical Autonomous Systemscited by 4
  9. 2025Rectified Robust Policy Optimization for Model-Uncertain Constrained Reinforcement Learning without Strong Dualitycited by 4
  10. 2023Learning Adaptive Safety for Multi-Agent Systemscited by 3
  11. 2024Feasibility Consistent Representation Learning for Safe Reinforcement Learningcited by 3
  12. 2024Enhancing Inverse Reinforcement Learning through Encoding Dynamic Information in Reward Shapingcited by 3
  13. 2024A Retention-Centric Framework for Continual Learning with Guaranteed Model Developmental Safetycited by 3
  14. 2024Flipping-based Policy for Chance-Constrained Markov Decision Processescited by 3
  15. 2024Certified Training with Branch-and-Bound for Lyapunov-stable Neural Controlcited by 3
  16. 2026Safe Flow Q-Learning: Offline Safe Reinforcement Learning with Reachability-Based Flow Policiescited by 3
  17. 2026Your Model Already Knows: Attention-Guided Safety Filter for Vision-Language-Action Modelscited by 3
  18. 2025AI Agents Should be Regulated Based on the Extent of Their Autonomous Operationscited by 2
  19. 2025Adaptive Shielding for Safe Reinforcement Learning under Hidden-Parameter Dynamics Shiftscited by 2
  20. 2026Joint Continual Learning of Local Language Models and Cloud Offloading Decisions with Budget Constraintscited by 2
  21. 2024Anytime-Constrained Equilibria in Polynomial Timecited by 1
  22. 2025Polynomial-Time Approximability of Constrained Reinforcement Learningcited by 1
  23. 2025Learning Vision-Based Neural Network Controllers with Semi-Probabilistic Safety Guaranteescited by 1
  24. 2025Belief-Based Offline Reinforcement Learning for Delay-Robust Policy Optimizationcited by 1
  25. 2025Accelerated Learning with Linear Temporal Logic using Differentiable Simulationcited by 1
  26. 2026Safe Continual Reinforcement Learning Methods for Nonstationary Environments. Towards a Survey of the State of the Artcited by 1
  27. 2026Dyna-Style Safety Augmented Reinforcement Learning: Staying Safe in the Face of Uncertaintycited by 1
  28. 2026SteinGate: Tail-Sensitive Safe Reinforcement Learning via Stein Discrepancycited by 1
  29. 2024Inverse Delayed Reinforcement Learningcited by 0
  30. 2025Navigating Demand Uncertainty in Container Shipping: Deep Reinforcement Learning for Enabling Adaptive and Feasible Master Stowage Planningcited by 0
  31. 2025Neural Lyapunov Function Approximation with Self-Supervised Reinforcement Learningcited by 0
  32. 2025Guardian: Decoupling Exploration from Safety in Reinforcement Learningcited by 0
  33. 2025Text-Conditioned Background Generation for Editable Multi-Layer Documentscited by 0
  34. 2025SB-TRPO: Towards Safe Reinforcement Learning with Hard Constraintscited by 0
  35. 2026Enhance the Safety in Reinforcement Learning by ADRC Lagrangian Methodscited by 0
  36. 2026Fuz-RL: A Fuzzy-Guided Robust Framework for Safe Reinforcement Learning under Uncertaintycited by 0
  37. 2026From Cumulative Constraints to Adaptive Runtime Safety Control for Nonstationary Reinforcement Learningcited by 0
  38. 2026Kernel-Based Safe Exploration in Deep Reinforcement Learningcited by 0

50 most cited of 54 All 54 (JSON)