# LyaNet: A Lyapunov Framework for Training Neural ODEs

arXiv: https://arxiv.org/abs/2202.02526 · HTML: https://mlsanity.com/paper/2202.02526/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 44 references that are in the map.
- First posted on arXiv 5 Feb 2022.

## Builds on (most cited first, 30 of 34)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Certified Adversarial Robustness via Randomized Smoothing](https://mlsanity.com/paper/1902.02918/) (2019)
- [Deep Equilibrium Models](https://mlsanity.com/paper/1909.01377/) (2019)
- [Provable defenses against adversarial examples via the convex outer adversarial polytope](https://mlsanity.com/paper/1711.00851/) (2017)
- [Augmented Neural ODEs](https://mlsanity.com/paper/1904.01681/) (2019)
- [Stable Architectures for Deep Neural Networks](https://mlsanity.com/paper/1705.03341/) (2017)
- [Certified Defenses against Adversarial Examples](https://mlsanity.com/paper/1801.09344/) (2018)
- [A Lyapunov-based Approach to Safe Reinforcement Learning](https://mlsanity.com/paper/1805.07708/) (2018)
- [Control Barrier Function Based Quadratic Programs for Safety Critical Systems](https://mlsanity.com/paper/1609.06408/) (2016)
- [Learning Agile Robotic Locomotion Skills by Imitating Animals](https://mlsanity.com/paper/2004.00784/) (2020)
- [Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equations](https://mlsanity.com/paper/1710.10121/) (2017)
- [Differentiable MPC for End-to-end Planning and Control](https://mlsanity.com/paper/1810.13400/) (2018)
- [Deep Neural Networks Motivated by Partial Differential Equations](https://mlsanity.com/paper/1804.04272/) (2018)
- [On the Sample Complexity of the Linear Quadratic Regulator](https://mlsanity.com/paper/1710.01688/) (2017)
- [Torchattacks: A PyTorch Repository for Adversarial Attacks](https://mlsanity.com/paper/2010.01950/) (2020)
- [Dissecting Neural ODEs](https://mlsanity.com/paper/2002.08071/) (2020)
- [Neural Lyapunov Control](https://mlsanity.com/paper/2005.00611/) (2020)
- [Learning Stable Deep Dynamics Models](https://mlsanity.com/paper/2001.06116/) (2020)
- [The Lyapunov Neural Network: Adaptive Stability Certification for Safe Learning of Dynamical Systems](https://mlsanity.com/paper/1808.00924/) (2018)
- [Stabilizing Equilibrium Models by Jacobian Regularization](https://mlsanity.com/paper/2106.14342/) (2021)
- [Moser Flow: Divergence-based Generative Modeling on Manifolds](https://mlsanity.com/paper/2108.08052/) (2021)
- [Continuous-in-Depth Neural Networks](https://mlsanity.com/paper/2008.02389/) (2020)
- [Learning Model Predictive Control for iterative tasks. A Data-Driven Control Framework](https://mlsanity.com/paper/1609.01387/) (2016)
- [Adversarial Robustness with Semi-Infinite Constrained Learning](https://mlsanity.com/paper/2110.15767/) (2021)
- [Control Regularization for Reduced Variance Reinforcement Learning](https://mlsanity.com/paper/1905.05380/) (2019)
- ["Hey, that's not an ODE": Faster ODE Adjoints via Seminorms](https://mlsanity.com/paper/2009.09457/) (2020)
- [Physics-Informed Neural Nets for Control of Dynamical Systems](https://mlsanity.com/paper/2104.02556/) (2021)
- [A Lyapunov Analysis of Momentum Methods in Optimization](https://mlsanity.com/paper/1611.02635/) (2016)

## Built on by (most cited first, 30 of 44)

- [Physics-Informed Machine Learning for Modeling and Control of Dynamical Systems](https://mlsanity.com/paper/2306.13867/) (2023)
- [Adversarial Robustness in Graph Neural Networks: A Hamiltonian Approach](https://mlsanity.com/paper/2310.06396/) (2023)
- [Learning Riemannian Stable Dynamical Systems via Diffeomorphisms](https://mlsanity.com/paper/2211.03169/) (2022)
- [Unleashing the Potential of Fractional Calculus in Graph Neural Networks with FROND](https://mlsanity.com/paper/2404.17099/) (2024)
- [Robust Classification using Contractive Hamiltonian Neural ODEs](https://mlsanity.com/paper/2203.11805/) (2022)
- [Lyapunov-Stable Deep Equilibrium Models](https://mlsanity.com/paper/2304.12707/) (2023)
- [Learning Robust State Observers using Neural ODEs (longer version)](https://mlsanity.com/paper/2212.00866/) (2022)
- [Adversarially Robust Out-of-Distribution Detection Using Lyapunov-Stabilized Embeddings](https://mlsanity.com/paper/2410.10744/) (2024)
- [Bounded nonlinear forecasts of partially observed geophysical systems with physics-constrained deep learning](https://mlsanity.com/paper/2202.05750/) (2022)
- [Robust Prototypical Few-Shot Organ Segmentation with Regularized Neural-ODEs](https://mlsanity.com/paper/2208.12428/) (2022)
- [DeNOTS: Stable Deep Neural ODEs for Time Series](https://mlsanity.com/paper/2408.08055/) (2024)
- [FI-ODE: Certifiably Robust Forward Invariance in Neural ODEs](https://mlsanity.com/paper/2210.16940/) (2022)
- [Efficient Certified Training and Robustness Verification of Neural ODEs](https://mlsanity.com/paper/2303.05246/) (2023)
- [Certified Robust Invariant Polytope Training in Neural Controlled ODEs](https://mlsanity.com/paper/2408.01273/) (2024)
- [Neural Predictive Control to Coordinate Discrete- and Continuous-Time Models for Time-Series Analysis with Control-Theoretical Improvements](https://mlsanity.com/paper/2508.01833/) (2025)
- [Lyapunov Probes for Hallucination Detection in Large Foundation Models](https://mlsanity.com/paper/2603.06081/) (2026)
- [Neural-Rendezvous: Provably Robust Guidance and Control to Encounter Interstellar Objects](https://mlsanity.com/paper/2208.04883/) (2022)
- [CTP:A Causal Interpretable Model for Non-Communicable Disease Progression Prediction](https://mlsanity.com/paper/2308.09735/) (2023)
- [Stable Autonomous Flow Matching](https://mlsanity.com/paper/2402.05774/) (2024)
- [Physics-constrained coupled neural differential equations for one dimensional blood flow modeling](https://mlsanity.com/paper/2411.05631/) (2024)
- [Deep Neural Networks Inspired by Differential Equations](https://mlsanity.com/paper/2510.09685/) (2025)
- [LyTimeT: Towards Robust and Interpretable State-Variable Discovery](https://mlsanity.com/paper/2510.19716/) (2025)
- [ICODE: Modeling Dynamical Systems with Extrinsic Input Information](https://mlsanity.com/paper/2411.13914/) (2024)
- [TANGO: Graph Neural Dynamics via Learned Energy and Tangential Flows](https://mlsanity.com/paper/2508.05070/) (2025)
- [Learning Generalized Hamiltonian Dynamics with Stability from Noisy Trajectory Data](https://mlsanity.com/paper/2509.07280/) (2025)
- [Random Spiking Neural Networks are Stable and Spectrally Simple](https://mlsanity.com/paper/2511.00904/) (2025)
- [Discovering Data Manifold Geometry through Geometric Properties](https://mlsanity.com/paper/2602.02611/) (2026)
- [Understanding the Theoretical Foundations of Deep Neural Networks through Differential Equations](https://mlsanity.com/paper/2603.18331/) (2026)
- [Lyapunov-Guided Training for Hardware-Safe Neural Networks Under Fixed-Point Arithmetic](https://mlsanity.com/paper/2607.04531/) (2026)
- [Noisy Learning for Neural ODEs Acts as a Robustness Locus Widening](https://mlsanity.com/paper/2206.08237/) (2022)

## Abstract (arXiv)

We propose a method for training ordinary differential equations by using a control-theoretic Lyapunov condition for stability. Our approach, called LyaNet, is based on a novel Lyapunov loss formulation that encourages the inference dynamics to converge quickly to the correct prediction. Theoretically, we show that minimizing Lyapunov loss guarantees exponential convergence to the correct solution and enables a novel robustness guarantee. We also provide practical algorithms, including one that avoids the cost of backpropagating through a solver or using the adjoint method. Relative to standard Neural ODE training, we empirically find that LyaNet can offer improved prediction performance, faster convergence of inference dynamics, and improved adversarial robustness. Our code available at https://github.com/ivandariojr/LyapunovLearning .