mlsanityData to 14 Oct 2026

Stability of Control Lyapunov Function Guided Reinforcement Learning

arXiv 2605.01978 ↗First posted 3 May 2026eess.SY cs.RO cs.SYcited by 0

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

Reinforcement learning (RL) has become the de facto method for achieving locomotion on humanoid robots in practice, yet stability analysis of the corresponding control policies is lacking. Recent work has attempted to merge control theoretic ideas with reinforcement learning through control guided learning. A notable example of this is the use of a control Lyapunov function (CLF) to synthesize the reinforcement learning rewards, a technique known as CLF-RL, which has shown practical success. This paper investigates the stability properties of optimal controllers using CLF-RL with the goal of bridging experimentally observed stability with theoretical guarantees. The RL problem is viewed as an optimal control problem and exponential stability is proven in both continuous and discrete time using both core CLF reward terms and the additional terms used in practice. The theoretical bounds are numerically verified on systems such as the double integrator and cart-pole. Finally, the CLF guided rewards are implemented for a walking humanoid robot to generate stable periodic orbits.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 10 of its 23 references that are in the map. First posted on arXiv 3 May 2026.

Builds on

10 of 23 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Chasing Stability: Humanoid Running via Control Lyapunov Function Guided Reinforcement Learning23 Sep
  2. 2025CLF-RL: Control Lyapunov Function Guided Reinforcement Learning12 Aug

Other references · 8

  1. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 512
  2. 2024Reinforcement learningcited by 247
  3. 2025BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusioncited by 165
  4. 2022Optimization-Based Control for Dynamic Legged Robotscited by 35
  5. 2022Lyapunov Design for Robust and Efficient Robotic Reinforcement Learningcited by 28
  6. 2026ZEST: Zero-shot Embodied Skill Transfer for Athletic Robot Controlcited by 11
  7. 2026Chasing Autonomy: Dynamic Retargeting and Control Guided RL for Performant and Controllable Humanoid Runningcited by 4
  8. 2014Relaxed ISS Small-Gain Theorems for Discrete-Time Systemscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Decomposing Control Lyapunov Functions for Efficient Reinforcement Learning
  2. 2025Stability Enhancement in Reinforcement Learning via Adaptive Control Lyapunov Function
  3. 2025Certifying Stability of Reinforcement Learning Policies using Generalized Lyapunov Functions
  4. 2025Neural Lyapunov Function Approximation with Self-Supervised Reinforcement Learning
  5. 2026Reinforcement Learning for Control with Probabilistic Stability Guarantee: A Finite-Sample Approach
  6. 2024Stable Inverse Reinforcement Learning: Policies from Control Lyapunov Landscapes
  7. 2024Revisiting Reward Design and Evaluation for Robust Humanoid Standing and Walking
  8. 2024Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control