# Training Neural ODEs Using Fully Discretized Simultaneous Optimization

arXiv: https://arxiv.org/abs/2502.15642 · HTML: https://mlsanity.com/paper/2502.15642/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 21 references that are in the map.
- First posted on arXiv 21 Feb 2025.

## Builds on (most cited first, 5 of 5)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [OMLT: Optimization & Machine Learning Toolkit](https://mlsanity.com/paper/2202.02414/) (2022)
- [SNODE: Spectral Discretization of Neural ODEs for System Identification](https://mlsanity.com/paper/1906.07038/) (2019)
- [Maximizing information from chemical engineering data sets: Applications to machine learning](https://mlsanity.com/paper/2201.10035/) (2022)
- [Accelerating Neural ODEs Using Model Order Reduction](https://mlsanity.com/paper/2105.14070/) (2021)

## Built on by (most cited first, 3 of 3)

- [A Simultaneous Approach for Training Neural Differential-Algebraic Systems of Equations](https://mlsanity.com/paper/2504.04665/) (2025)
- [Efficient Training of Physics-enhanced Neural ODEs via Direct Collocation and Nonlinear Programming](https://mlsanity.com/paper/2505.03552/) (2025)
- [Meta-learning for sample-efficient Bayesian optimisation of fed-batch processes](https://mlsanity.com/paper/2605.05382/) (2026)

## Abstract (arXiv)

Neural Ordinary Differential Equations (Neural ODEs) represent continuous-time dynamics with neural networks, offering advancements for modeling and control tasks. However, training Neural ODEs requires solving differential equations at each epoch, leading to high computational costs. This work investigates simultaneous optimization methods as a faster training alternative. In particular, we employ a collocation-based, fully discretized formulation and use IPOPT--a solver for large-scale nonlinear optimization--to simultaneously optimize collocation coefficients and neural network parameters. Using the Van der Pol Oscillator as a case study, we demonstrate faster convergence compared to traditional training methods. Furthermore, we introduce a decomposition framework utilizing Alternating Direction Method of Multipliers (ADMM) to effectively coordinate sub-models among data batches. Our results show significant potential for (collocation-based) simultaneous Neural ODE training pipelines.