mlsanityData to 14 Oct 2026

A Review On Safe Reinforcement Learning Using Lyapunov and Barrier Functions

arXiv 2508.09128 ↗First posted 12 Aug 2025eess.SY cs.SYcited by 4

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

Reinforcement learning (RL) has proven to be particularly effective in solving complex decision-making problems for a wide range of applications. Safe reinforcement learning refers to a class of constrained problems where the constraint violations lead to partial or complete system failure. The goal of this review is to provide an overview of safe RL techniques using Lyapunov and barrier functions to guarantee this notion of safety (stability of the system in terms of a computed policy and constraint satisfaction during training and deployment). Three concrete takeaways emerge from our analysis: (i) the field has shifted decisively from model-based to model-free formulations since 2017, with combined CLF-CBF approaches becoming the most active sub-area post-2022; (ii) per-class open problems are now well-defined, certificate validity under function approximation and distribution shift for Lyapunov methods, feasibility and deadlock under hard CBF-QP shielding for barrier methods, and joint CLF--CBF feasibility under model uncertainty for combined methods; and (iii) deployment to high-dimensional and partially observable settings remains the dominant scalability barrier across all three classes. The different approaches employed are discussed in detail along with their shortcomings and benefits to provide critique and possible future research directions. The review demonstrates promising scope for providing safety guarantees for complex dynamical systems with operational constraints using model-based and model-free RL.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 65 of its 145 references that are in the map. First posted on arXiv 12 Aug 2025.

Builds on

65 of 145 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Certificated Actor-Critic: Hierarchical Reinforcement Learning with Control Barrier Functions for Safe Navigation29 Jan
  2. 2024Disturbance Observer-based Control Barrier Functions with Residual Model Learning for Safe Reinforcement Learning9 Oct
  3. 2024Formally Verifying Deep Reinforcement Learning Controllers with Lyapunov Barrier Certificates22 May
  4. 2024Reinforcement Learning-based Receding Horizon Control using Adaptive Control Barrier Functions for Safety-Critical Systems26 Mar
  5. 2024Constrained Reinforcement Learning with Smoothed Log Barrier Function21 Mar

Other references · 60

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  2. 2016OpenAI Gymcited by 1,736
  3. 2017Deep Reinforcement Learning that Matterscited by 754
  4. 2019Model-Based Reinforcement Learning for Ataricited by 502
  5. 2017A Brief Survey of Deep Reinforcement Learningcited by 483
  6. 2017Safe Reinforcement Learning via Shieldingcited by 449
  7. 2019Control Barrier Functions: Theory and Applicationscited by 372
  8. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 371
  9. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 366
  10. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 297
  11. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 289
  12. 2016Control Barrier Function Based Quadratic Programs for Safety Critical Systemscited by 287
38 more
  1. 2021Modern Koopman Theory for Dynamical Systemscited by 276
  2. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 213
  3. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 181
  4. 2022Safe Control with Learned Certificates: A Survey of Neural Lyapunov, Barrier, and Contraction methodscited by 151
  5. 2019Lyapunov-based Safe Policy Optimization for Continuous Controlcited by 145
  6. 2023Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmarkcited by 143
  7. 2017AI Safety Gridworldscited by 110
  8. 2019Safe Reinforcement Learning Using Robust MPCcited by 85
  9. 2020Reinforcement Learning for Safety-Critical Control under Model Uncertainty, using Control Lyapunov Functions and Control Barrier Functionscited by 68
  10. 2021Safe Multi-Agent Reinforcement Learning via Shieldingcited by 61
  11. 2022Enforcing Hard Constraints with Soft Barriers: Safe Reinforcement Learning in Unknown Stochastic Environmentscited by 60
  12. 2021Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challengescited by 50
  13. 2021safe-control-gym: a Unified Benchmark Suite for Safe Learning-based Control and Reinforcement Learning in Roboticscited by 50
  14. 2021Deep Reinforcement Learning in Computer Vision: A Comprehensive Surveycited by 43
  15. 2018DQN-TAMER: Human-in-the-Loop Reinforcement Learning with Intractable Feedbackcited by 41
  16. 2023Tuning computer vision models with task rewardscited by 37
  17. 2021Stabilizing Neural Control Using Self-Learned Almost Lyapunov Criticscited by 35
  18. 2024CoBL-Diffusion: Diffusion-Based Conditional Robot Planning in Dynamic Environments Using Control Barrier and Lyapunov Functionscited by 33
  19. 2021Model-based Constrained Reinforcement Learning using Generalized Control Barrier Functioncited by 32
  20. 2022i-Sim2Real: Reinforcement Learning of Robotic Policies in Tight Human-Robot Interaction Loopscited by 32
  21. 2022Lyapunov Design for Robust and Efficient Robotic Reinforcement Learningcited by 28
  22. 2019Temporal Logic Guided Safe Reinforcement Learning Using Control Barrier Functionscited by 21
  23. 2021Reinforcement Learning for Feedback-Enabled Cyber Resiliencecited by 19
  24. 2018Barrier-Certified Adaptive Reinforcement Learning with Applications to Brushbot Navigationcited by 18
  25. 2022Distributed Reinforcement Learning for Robot Teams: A Reviewcited by 16
  26. 2020Safe reinforcement learning for probabilistic reachability and safety specifications: A Lyapunov-based approachcited by 13
  27. 2022Evaluating Vision Transformer Methods for Deep Reinforcement Learning from Pixelscited by 13
  28. 2023Stable and Safe Reinforcement Learning via a Barrier-Lyapunov Actor-Critic Approachcited by 13
  29. 2023Reinforcement Learning for Safe Robot Control using Control Lyapunov Barrier Functionscited by 13
  30. 2021Safe Exploration in Model-based Reinforcement Learning using Control Barrier Functionscited by 11
  31. 2022Safe and Efficient Reinforcement Learning Using Disturbance-Observer-Based Control Barrier Functionscited by 11
  32. 2024Barrier Functions Inspired Reward Shaping for Reinforcement Learningcited by 11
  33. 2021Safe Multi-Agent Reinforcement Learning through Decentralized Multiple Control Barrier Functionscited by 10
  34. 2021Barrier Function-based Safe Reinforcement Learning for Emergency Control of Power Systemscited by 8
  35. 2024Actor-Critic Physics-informed Neural Lyapunov Controlcited by 8
  36. 2023Finite Time Lyapunov Exponent Analysis of Model Predictive Control and Reinforcement Learningcited by 7
  37. 2022Safe Reinforcement Learning via Confidence-Based Filterscited by 6
  38. 2020Learning Min-norm Stabilizing Control Laws for Systems with Unknown Dynamicscited by 5

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026LC-SAC: Lyapunov-Constrained Soft Actor-Critic via Koopman Operator Theory for Trajectory Tracking and Stabilizationcited by 1
  2. 2026Robust Koopman Control Barrier Filters for Safe Actor-Critic Reinforcement Learningcited by 0
  3. 2026CaLiSym: Learning Symplectic Dynamics of Real-World Systems through Structured Canonical Liftscited by 0
  4. 2026Safe Greenhouse Climate Control Using Lagrangian-Constrained PPO with Kolmogorov-Arnold Networkscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Multi-Constraint Safe Reinforcement Learning via Closed-form Solution for Log-Sum-Exp Approximation of Control Barrier Functions
  2. 2025CBF-RL: Safety Filtering Reinforcement Learning in Training with Control Barrier Functions
  3. 2025Safe model-based Reinforcement Learning via Model Predictive Control and Control Barrier Functions
  4. 2024A Survey of Constraint Formulations in Safe Reinforcement Learning
  5. 2024Learning Control Barrier Functions and their application in Reinforcement Learning: A Survey
  6. 2026CAPSULE: Control-Theoretic Action Perturbations for Safe Uncertainty-Aware Reinforcement Learning
  7. 2025Stability Enhancement in Reinforcement Learning via Adaptive Control Lyapunov Function
  8. 2025Safe Learning Control with Optimality and Stability Guarantees