# LiLaN: A Linear Latent Network as the Solution Operator for Real-Time Solutions to Stiff Nonlinear Ordinary Differential Equations

arXiv: https://arxiv.org/abs/2501.08423 · HTML: https://mlsanity.com/paper/2501.08423/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 42 references that are in the map.
- First posted on arXiv 14 Jan 2025.

## Builds on (most cited first, 20 of 20)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Deep learning for universal linear embeddings of nonlinear dynamics](https://mlsanity.com/paper/1712.09707/) (2017)
- [Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1711.10561/) (2017)
- [On Neural Differential Equations](https://mlsanity.com/paper/2202.02435/) (2022)
- [Universal Approximation with Deep Narrow Networks](https://mlsanity.com/paper/1905.08539/) (2019)
- [Understanding and mitigating gradient pathologies in physics-informed neural networks](https://mlsanity.com/paper/2001.04536/) (2020)
- [Forecasting Sequential Data using Consistent Koopman Autoencoders](https://mlsanity.com/paper/2003.02236/) (2020)
- [Stiff Neural Ordinary Differential Equations](https://mlsanity.com/paper/2103.15341/) (2021)
- [Parameterized Neural Ordinary Differential Equations: Applications to Computational Physics Problems](https://mlsanity.com/paper/2010.14685/) (2020)
- [Can Increasing Input Dimensionality Improve Deep Reinforcement Learning?](https://mlsanity.com/paper/2003.01629/) (2020)
- [Expressivity of Deep Neural Networks](https://mlsanity.com/paper/2007.04759/) (2020)
- [Learning stiff chemical kinetics using extended deep neural operators](https://mlsanity.com/paper/2302.12645/) (2023)
- [A Survey on Universal Approximation Theorems](https://mlsanity.com/paper/2407.12895/) (2024)
- [Accelerating Simulation of Stiff Nonlinear Systems using Continuous-Time Echo State Networks](https://mlsanity.com/paper/2010.04004/) (2020)
- [Speeding up astrochemical reaction networks with autoencoders and neural ODEs](https://mlsanity.com/paper/2312.06015/) (2023)
- [A Unified and Constructive Framework for the Universality of Neural Networks](https://mlsanity.com/paper/2112.14877/) (2021)
- [A Model-Constrained Tangent Slope Learning Approach for Dynamical Systems](https://mlsanity.com/paper/2208.04995/) (2022)
- [Reduced-PINN: An Integration-Based Physics-Informed Neural Networks for Stiff ODEs](https://mlsanity.com/paper/2208.12045/) (2022)
- [Numerical Solution of Stiff ODEs with Physics-Informed RPNNs](https://mlsanity.com/paper/2108.01584/) (2021)
- [A Model-Constrained Discontinuous Galerkin Network (DGNet) for Compressible Euler Equations with Out-of-Distribution Generalization](https://mlsanity.com/paper/2409.18371/) (2024)

## Abstract (arXiv)

Solving stiff ordinary differential equations (StODEs) requires sophisticated numerical solvers, which are often computationally expensive. In general, traditional explicit time integration schemes with restricted time step sizes are not suitable for StODEs, and one must resort to costly implicit methods. On the other hand, state-of-the-art machine learning based methods, such as Neural ODE, poorly handle the timescale separation of various elements of the solutions to StODEs, while still requiring expensive implicit/explicit integration at inference time. In this work, we propose a linear latent network (LiLaN) approach in which the dynamics in the latent space can be integrated analytically, and thus numerical integration is completely avoided. At the heart of LiLaN are the following key ideas: i) two encoder networks to encode the initial condition together with parameters of the ODE to the slope and the initial condition for the latent dynamics, respectively. Since the latent dynamics, by design, are linear, the solution can be evaluated analytically; ii) a neural network to map the physical time to latent times, one for each latent variable. Finally, iii) a decoder network to decode the latent solution to the physical solution at the corresponding physical time. We provide a universal approximation theorem for the proposed LiLaN approach, showing that it can approximate the solution of any stiff nonlinear system on a compact set to any degree of accuracy epsilon. We also show an interesting fact that the dimension of the latent dynamical system in LiLaN is independent of epsilon. Numerical results on the "Robertson Stiff Chemical Kinetics Model," "Plasma Collisional-Radiative Model," and "Allen-Cahn" and "Cahn-Hilliard" PDEs suggest that LiLaN outperformed state-of-the-art machine learning approaches for handling stiff ordinary and partial differential equations.