# Taylor-Lagrange Neural Ordinary Differential Equations: Toward Fast Training and Evaluation of Neural ODEs

arXiv: https://arxiv.org/abs/2201.05715 · HTML: https://mlsanity.com/paper/2201.05715/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 36 references that are in the map.
- First posted on arXiv 14 Jan 2022.

## Builds on (most cited first, 18 of 18)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [Masked Autoregressive Flow for Density Estimation](https://mlsanity.com/paper/1705.07057/) (2017)
- [FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models](https://mlsanity.com/paper/1810.01367/) (2018)
- [Lagrangian Neural Networks](https://mlsanity.com/paper/2003.04630/) (2020)
- [How to train your neural ODE: the world of Jacobian and kinetic regularization](https://mlsanity.com/paper/2002.02798/) (2020)
- [Riemannian Continuous Normalizing Flows](https://mlsanity.com/paper/2006.10605/) (2020)
- [Simplifying Hamiltonian and Lagrangian Neural Networks via Explicit Constraints](https://mlsanity.com/paper/2010.13581/) (2020)
- [Learning Differential Equations that are Easy to Solve](https://mlsanity.com/paper/2007.04504/) (2020)
- [STEER: Simple Temporal Regularization For Neural ODEs](https://mlsanity.com/paper/2006.10711/) (2020)
- [Neural Networks with Physics-Informed Architectures and Constraints for Dynamical Systems Modeling](https://mlsanity.com/paper/2109.06407/) (2021)
- [Benchmarking Energy-Conserving Neural Networks for Learning Dynamics from Data](https://mlsanity.com/paper/2012.02334/) (2020)
- [Hypersolvers: Toward Fast Continuous-Depth Models](https://mlsanity.com/paper/2007.09601/) (2020)
- [Opening the Blackbox: Accelerating Neural Differential Equations by Regularizing Internal Solver Heuristics](https://mlsanity.com/paper/2105.03918/) (2021)
- [Structured Mechanical Models for Robot Learning and Control](https://mlsanity.com/paper/2004.10301/) (2020)
- [Deep Diffeomorphic Normalizing Flows](https://mlsanity.com/paper/1810.03256/) (2018)
- [On-The-Fly Control of Unknown Smooth Systems from Limited Data](https://mlsanity.com/paper/2009.12733/) (2020)

## Built on by (most cited first, 13 of 13)

- [GENIE: Higher-Order Denoising Diffusion Solvers](https://mlsanity.com/paper/2210.05475/) (2022)
- [Compositional Learning of Dynamical System Models Using Port-Hamiltonian Neural Networks](https://mlsanity.com/paper/2212.00893/) (2022)
- [High-Order Matching for One-Step Shortcut Diffusion Models](https://mlsanity.com/paper/2502.00688/) (2025)
- [Theoretical Guarantees for High Order Trajectory Refinement in Generative Flows](https://mlsanity.com/paper/2503.09069/) (2025)
- [Learning Robust State Observers using Neural ODEs (longer version)](https://mlsanity.com/paper/2212.00866/) (2022)
- [Autonomous Drifting with 3 Minutes of Data via Learned Tire Models](https://mlsanity.com/paper/2306.06330/) (2023)
- [Continuous Deep Equilibrium Models: Training Neural ODEs faster by integrating them to Infinity](https://mlsanity.com/paper/2201.12240/) (2022)
- [Learning Subgrid-scale Models with Neural Ordinary Differential Equations](https://mlsanity.com/paper/2212.09967/) (2022)
- [Locally Regularized Neural Differential Equations: Some Black Boxes Were Meant to Remain Closed!](https://mlsanity.com/paper/2303.02262/) (2023)
- [HOFAR: High-Order Augmentation of Flow Autoregressive Transformers](https://mlsanity.com/paper/2503.08032/) (2025)
- [Noisy Learning for Neural ODEs Acts as a Robustness Locus Widening](https://mlsanity.com/paper/2206.08237/) (2022)
- [Improved Batching Strategy For Irregular Time-Series ODE](https://mlsanity.com/paper/2207.05708/) (2022)
- [Differentiable DG with Neural Operator Source Term Correction](https://mlsanity.com/paper/2310.18897/) (2023)

## Abstract (arXiv)

Neural ordinary differential equations (NODEs) -- parametrizations of differential equations using neural networks -- have shown tremendous promise in learning models of unknown continuous-time dynamical systems from data. However, every forward evaluation of a NODE requires numerical integration of the neural network used to capture the system dynamics, making their training prohibitively expensive. Existing works rely on off-the-shelf adaptive step-size numerical integration schemes, which often require an excessive number of evaluations of the underlying dynamics network to obtain sufficient accuracy for training. By contrast, we accelerate the evaluation and the training of NODEs by proposing a data-driven approach to their numerical integration. The proposed Taylor-Lagrange NODEs (TL-NODEs) use a fixed-order Taylor expansion for numerical integration, while also learning to estimate the expansion's approximation error. As a result, the proposed approach achieves the same accuracy as adaptive step-size schemes while employing only low-order Taylor expansions, thus greatly reducing the computational cost necessary to integrate the NODE. A suite of numerical experiments, including modeling dynamical systems, image classification, and density estimation, demonstrate that TL-NODEs can be trained more than an order of magnitude faster than state-of-the-art approaches, without any loss in performance.