mlsanityData to 7 Oct 2026

Critic as Lyapunov function (CALF): a model-free, stability-ensuring agent

arXiv 2409.09869 ↗First posted 15 Sep 2024cs.RO cs.AI math.OCcited by 4

Not on the map. Why?

Abstract (arXiv)

This work presents and showcases a novel reinforcement learning agent called Critic As Lyapunov Function (CALF) which is model-free and ensures online environment, in other words, dynamical system stabilization. Online means that in each learning episode, the said environment is stabilized. This, as demonstrated in a case study with a mobile robot simulator, greatly improves the overall learning performance. The base actor-critic scheme of CALF is analogous to SARSA. The latter did not show any success in reaching the target in our studies. However, a modified version thereof, called SARSA-m here, did succeed in some learning scenarios. Still, CALF greatly outperformed the said approach. CALF was also demonstrated to improve a nominal stabilizer provided to it. In summary, the presented agent may be considered a viable approach to fusing classical control with reinforcement learning. Its concurrent approaches are mostly either offline or model-based, like, for instance, those that fuse model-predictive control into the agent.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 24 of its 56 references that are in the map. First posted on arXiv 15 Sep 2024.

Builds on

24 of 56 references.

References · 24

  1. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  2. 2022DayDreamer: World Models for Physical Robot Learningcited by 372
  3. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 347
  4. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 278
  5. 2016Deep Visual Foresight for Planning Robot Motioncited by 276
  6. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 256
  7. 2018Differentiable MPC for End-to-end Planning and Controlcited by 178
  8. 2020Recovery RL: Safe Reinforcement Learning with Learned Recovery Zonescited by 156
  9. 2018Learning-based Model Predictive Control for Safe Explorationcited by 116
  10. 2018Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based Controlcited by 104
  11. 2017Trial without Error: Towards Safe Reinforcement Learning via Human Interventioncited by 102
  12. 2019Safe Reinforcement Learning Using Robust MPCcited by 72
12 more
  1. 2020Reinforcement Learning for Safety-Critical Control under Model Uncertainty, using Control Lyapunov Functions and Control Barrier Functionscited by 61
  2. 2019Safe Reinforcement Learning on Autonomous Vehiclescited by 48
  3. 2019Learning Human Objectives by Evaluating Hypothetical Behaviorcited by 41
  4. 2020Actor-Critic Reinforcement Learning for Control with Stability Guaranteecited by 39
  5. 2021Stabilizing Neural Control Using Self-Learned Almost Lyapunov Criticscited by 33
  6. 2020Infinite-Horizon Differentiable Model Predictive Controlcited by 16
  7. 2019Practical Reinforcement Learning of Stabilizing Economic MPCcited by 15
  8. 2020Practical Reinforcement Learning For MPC: Learning from sparse objectives in under an hour on a real robotcited by 15
  9. 2019Deep Value Model Predictive Controlcited by 14
  10. 2019Fixed-Horizon Temporal Difference Methods for Stable Reinforcement Learningcited by 12
  11. 2020Deductive Stability Proofs for Ordinary Differential Equationscited by 3
  12. 2023Transferring Multiple Policies to Hotstart Reinforcement Learning in an Air Compressor Management Problemcited by 1

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 3.

4 papers.

  1. 2025Off Policy Lyapunov Stability in Reinforcement Learningcited by 2
  2. 2024A novel agent with formal goal-reaching guarantees: an experimental study with a mobile robotcited by 0
  3. 2025Multi-CALF: A Policy Combination Approach with Statistical Guaranteescited by 0
  4. 2025A universal policy wrapper with guaranteescited by 0