# Stabilizing Neural Control Using Self-Learned Almost Lyapunov Critics

arXiv: https://arxiv.org/abs/2107.04989 · HTML: https://mlsanity.com/paper/2107.04989/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jul 2021.

## Built on by (most cited first, 30 of 35)

- [Iterative Reachability Estimation for Safe Reinforcement Learning](https://mlsanity.com/paper/2309.13528/) (2023)
- [Learning Control Policies for Stochastic Systems with Reach-avoid Guarantees](https://mlsanity.com/paper/2210.05308/) (2022)
- [Neural Lyapunov Control for Discrete-Time Systems](https://mlsanity.com/paper/2305.06547/) (2023)
- [Formally Verifying Deep Reinforcement Learning Controllers with Lyapunov Barrier Certificates](https://mlsanity.com/paper/2405.14058/) (2024)
- [Hamilton-Jacobi Reachability in Reinforcement Learning: A Survey](https://mlsanity.com/paper/2407.09645/) (2024)
- [Policy Optimization with Advantage Regularization for Long-Term Fairness in Decision Systems](https://mlsanity.com/paper/2210.12546/) (2022)
- [Compositional Policy Learning in Stochastic Control Systems with Formal Guarantees](https://mlsanity.com/paper/2312.01456/) (2023)
- [Stable and Safe Reinforcement Learning via a Barrier-Lyapunov Actor-Critic Approach](https://mlsanity.com/paper/2304.04066/) (2023)
- [Physics-Regulated Deep Reinforcement Learning: Invariant Embeddings](https://mlsanity.com/paper/2305.16614/) (2023)
- [DexCatch: Learning to Catch Arbitrary Objects with Dexterous Hands](https://mlsanity.com/paper/2310.08809/) (2023)
- [Total Energy Shaping with Neural Interconnection and Damping Assignment -- Passivity Based Control](https://mlsanity.com/paper/2112.12999/) (2021)
- [Learning Provably Stabilizing Neural Controllers for Discrete-Time Stochastic Systems](https://mlsanity.com/paper/2210.05304/) (2022)
- [Learning Control-Oriented Dynamical Structure from Data](https://mlsanity.com/paper/2302.02529/) (2023)
- [Safe and Reliable Training of Learning-Based Aerospace Controllers](https://mlsanity.com/paper/2407.07088/) (2024)
- [Critic as Lyapunov function (CALF): a model-free, stability-ensuring agent](https://mlsanity.com/paper/2409.09869/) (2024)
- [A Review On Safe Reinforcement Learning Using Lyapunov and Barrier Functions](https://mlsanity.com/paper/2508.09128/) (2025)
- [Certifiably Robust Reinforcement Learning through Model-Based Abstract Interpretation](https://mlsanity.com/paper/2301.11374/) (2023)
- [Physical Deep Reinforcement Learning Towards Safety Guarantee](https://mlsanity.com/paper/2303.16860/) (2023)
- [Stabilizing reinforcement learning control: A modular framework for optimizing over all stable behavior](https://mlsanity.com/paper/2310.14098/) (2023)
- [Simplex-enabled Safe Continual Learning Machine](https://mlsanity.com/paper/2409.05898/) (2024)
- [Reinforcement Learning-based Control via Y-wise Affine Neural Networks (YANNs)](https://mlsanity.com/paper/2508.16474/) (2025)
- [Off Policy Lyapunov Stability in Reinforcement Learning](https://mlsanity.com/paper/2509.09863/) (2025)
- [A Policy Optimization Method Towards Optimal-time Stability](https://mlsanity.com/paper/2301.00521/) (2023)
- [Extremum-Seeking Action Selection for Accelerating Policy Optimization](https://mlsanity.com/paper/2404.01598/) (2024)
- [A novel agent with formal goal-reaching guarantees: an experimental study with a mobile robot](https://mlsanity.com/paper/2409.14867/) (2024)
- [Learn Proportional Derivative Controllable Latent Space from Pixels](https://mlsanity.com/paper/2110.08239/) (2021)
- [Efficient Exploration Using Extra Safety Budget in Constrained Policy Optimization](https://mlsanity.com/paper/2302.14339/) (2023)
- [Autonomous Wheel Loader Navigation Using Goal-Conditioned Actor-Critic MPC](https://mlsanity.com/paper/2409.15717/) (2024)
- [Physics-model-guided Worst-case Sampling for Safe Reinforcement Learning](https://mlsanity.com/paper/2412.13224/) (2024)
- [Is Bellman Equation Enough for Learning Control?](https://mlsanity.com/paper/2503.02171/) (2025)

## Abstract (arXiv)

The lack of stability guarantee restricts the practical use of learning-based methods in core control problems in robotics. We develop new methods for learning neural control policies and neural Lyapunov critic functions in the model-free reinforcement learning (RL) setting. We use sample-based approaches and the Almost Lyapunov function conditions to estimate the region of attraction and invariance properties through the learned Lyapunov critic functions. The methods enhance stability of neural controllers for various nonlinear systems including automobile and quadrotor control.