mlsanityData to 14 Oct 2026

Stability Enhancement in Reinforcement Learning via Adaptive Control Lyapunov Function

arXiv 2504.19473 ↗First posted 18 Jan 2025cs.LG cs.ROcited by 1

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

Reinforcement Learning (RL) has shown promise in control tasks but faces significant challenges in real-world applications, primarily due to the absence of safety guarantees during the learning process. Existing methods often struggle with ensuring safe exploration, leading to potential system failures and restricting applications primarily to simulated environments. Traditional approaches such as reward shaping and constrained policy optimization can fail to guarantee safety during initial learning stages, while model-based methods using Control Lyapunov Functions (CLFs) or Control Barrier Functions (CBFs) may hinder efficient exploration and performance. To address these limitations, this paper introduces Soft Actor-Critic with Control Lyapunov Function (SAC-CLF), a framework that enhances stability and safety through three key innovations: (1) a task-specific CLF design method for safe and optimal performance; (2) dynamic adjustment of constraints to maintain robustness under unmodeled dynamics; and (3) improved control input smoothness while ensuring safety. Experimental results on a classical nonlinear system and satellite attitude control demonstrate the effectiveness of SAC-CLF in overcoming the shortcomings of existing methods.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 13 of its 32 references that are in the map. First posted on arXiv 18 Jan 2025.

Builds on

13 of 32 references.

References · 13

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2015Continuous control with deep reinforcement learningcited by 3,407
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  4. 2015Trust Region Policy Optimizationcited by 2,580
  5. 2017AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehiclescited by 523
  6. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 413
  7. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 366
  8. 2016Control Barrier Function Based Quadratic Programs for Safety Critical Systemscited by 287
  9. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 181
  10. 2017A General Safety Framework for Learning-Based Control in Uncertain Robotic Systemscited by 163
  11. 2022Safe Reinforcement Learning for Legged Locomotioncited by 35
  12. 2013Torque Saturation in Bipedal Robotic Walking through Control Lyapunov Function Based Quadratic Programscited by 3
1 more
  1. 2024DeepSafeMPC: Deep Learning-Based Model Predictive Control for Safe Multi-Agent Reinforcement Learningcited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025MSACL: Multi-Step Actor-Critic Learning with Lyapunov Certificates for Exponentially Stabilizing Controlcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Review On Safe Reinforcement Learning Using Lyapunov and Barrier Functions
  2. 2025Safe Learning Control with Optimality and Stability Guarantees
  3. 2024Safe Deep Model-Based Reinforcement Learning with Lyapunov Functions
  4. 2025Certifying Stability of Reinforcement Learning Policies using Generalized Lyapunov Functions
  5. 2026Stability of Control Lyapunov Function Guided Reinforcement Learning
  6. 2024Decomposing Control Lyapunov Functions for Efficient Reinforcement Learning
  7. 2026LC-SAC: Lyapunov-Constrained Soft Actor-Critic via Koopman Operator Theory for Trajectory Tracking and Stabilization
  8. 2026Learning over Forward-Invariant Policy Classes: Reinforcement Learning without Safety Concerns