# Actor-Critic Physics-informed Neural Lyapunov Control

arXiv: https://arxiv.org/abs/2403.08448 · HTML: https://mlsanity.com/paper/2403.08448/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 18 of its 38 references that are in the map.
- First posted on arXiv 13 Mar 2024.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 18 of 18)

- [Differentiable Convex Optimization Layers](https://mlsanity.com/paper/1910.12430/) (2019)
- [Control Barrier Function Based Quadratic Programs for Safety Critical Systems](https://mlsanity.com/paper/1609.06408/) (2016)
- [Safe Control with Learned Certificates: A Survey of Neural Lyapunov, Barrier, and Contraction methods](https://mlsanity.com/paper/2202.11762/) (2022)
- [Neural Lyapunov Control](https://mlsanity.com/paper/2005.00611/) (2020)
- [Learning Stability Certificates from Data](https://mlsanity.com/paper/2008.05952/) (2020)
- [Neural Lyapunov Control of Unknown Nonlinear Systems with Stability Guarantees](https://mlsanity.com/paper/2206.01913/) (2022)
- [Neural Certificates for Safe Control Policies](https://mlsanity.com/paper/2006.08465/) (2020)
- [Neural Lyapunov Control for Discrete-Time Systems](https://mlsanity.com/paper/2305.06547/) (2023)
- [Physics-Informed Neural Network Lyapunov Functions: PDE Characterization, Learning, and Verification](https://mlsanity.com/paper/2312.09131/) (2023)
- [Lyapunov-Net: A Deep Neural Network Architecture for Lyapunov Function Approximation](https://mlsanity.com/paper/2109.13359/) (2021)
- [Computing Lyapunov functions using deep neural networks](https://mlsanity.com/paper/2005.08965/) (2020)
- [Physics-Informed Neural Network Policy Iteration: Algorithms, Convergence, and Verification](https://mlsanity.com/paper/2402.10119/) (2024)
- [Some Applications of Polynomial Optimization in Operations Research and Real-Time Decision Making](https://mlsanity.com/paper/1504.06002/) (2015)
- [Aggressive Online Control of a Quadrotor via Deep Network Representations of Optimality Principles](https://mlsanity.com/paper/1912.07067/) (2019)
- [Automated and Sound Synthesis of Lyapunov Functions with SMT Solvers](https://mlsanity.com/paper/2007.10865/) (2020)
- [Neural Stochastic Control](https://mlsanity.com/paper/2209.07240/) (2022)
- [Data-Driven Computational Methods for the Domain of Attraction and Zubov's Equation](https://mlsanity.com/paper/2112.14415/) (2021)
- [Convex Incremental Dissipativity Analysis of Nonlinear Systems - Extended version](https://mlsanity.com/paper/2006.14201/) (2020)

## Built on by (most cited first, 8 of 8)

- [Two-Stage Learning of Stabilizing Neural Controllers via Zubov Sampling and Iterative Domain Expansion](https://mlsanity.com/paper/2506.01356/) (2025)
- [Neural Contraction Metrics with Formal Guarantees for Discrete-Time Nonlinear Dynamical Systems](https://mlsanity.com/paper/2504.17102/) (2025)
- [A Review On Safe Reinforcement Learning Using Lyapunov and Barrier Functions](https://mlsanity.com/paper/2508.09128/) (2025)
- [Safe Deployment of Offline Reinforcement Learning via Input Convex Action Correction](https://mlsanity.com/paper/2507.22640/) (2025)
- [Bridging Control with Neural Network Verifier alpha-beta-CROWN: A Tutorial](https://mlsanity.com/paper/2605.26577/) (2026)
- [Coordinating Planning and Tracking in Layered Control Policies via Actor-Critic Learning](https://mlsanity.com/paper/2408.01639/) (2024)
- [(Un)supervised Learning of Maximal Lyapunov Functions](https://mlsanity.com/paper/2408.17246/) (2024)
- [Backstepping-Guided Reinforcement Learning for Wide-Range Saint-Venant Canal Regulation](https://mlsanity.com/paper/2608.20089/) (2026)

## Abstract (arXiv)

Designing control policies for stabilization tasks with provable guarantees is a long-standing problem in nonlinear control. A crucial performance metric is the size of the resulting region of attraction, which essentially serves as a robustness "margin" of the closed-loop system against uncertainties. In this paper, we propose a new method to train a stabilizing neural network controller along with its corresponding Lyapunov certificate, aiming to maximize the resulting region of attraction while respecting the actuation constraints. Crucial to our approach is the use of Zubov's Partial Differential Equation (PDE), which precisely characterizes the true region of attraction of a given control policy. Our framework follows an actor-critic pattern where we alternate between improving the control policy (actor) and learning a Zubov function (critic). Finally, we compute the largest certifiable region of attraction by invoking an SMT solver after the training procedure. Our numerical experiments on several design problems show consistent and significant improvements in the size of the resulting region of attraction.