# torchode: A Parallel ODE Solver for PyTorch

arXiv: https://arxiv.org/abs/2210.12375 · HTML: https://mlsanity.com/paper/2210.12375/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 29 references that are in the map.
- First posted on arXiv 22 Oct 2022.

## Builds on (most cited first, 18 of 18)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models](https://mlsanity.com/paper/1810.01367/) (2018)
- [Neural Controlled Differential Equations for Irregular Time Series](https://mlsanity.com/paper/2005.08926/) (2020)
- [Augmented Neural ODEs](https://mlsanity.com/paper/1904.01681/) (2019)
- [On Neural Differential Equations](https://mlsanity.com/paper/2202.02435/) (2022)
- [GRU-ODE-Bayes: Continuous modeling of sporadically-observed time series](https://mlsanity.com/paper/1905.12374/) (2019)
- [How to train your neural ODE: the world of Jacobian and kinetic regularization](https://mlsanity.com/paper/2002.02798/) (2020)
- [Graph Neural Ordinary Differential Equations](https://mlsanity.com/paper/1911.07532/) (2019)
- [Dissecting Neural ODEs](https://mlsanity.com/paper/2002.08071/) (2020)
- [ANODE: Unconditionally Accurate Memory-Efficient Gradients for Neural ODEs](https://mlsanity.com/paper/1902.10298/) (2019)
- [STEER: Simple Temporal Regularization For Neural ODEs](https://mlsanity.com/paper/2006.10711/) (2020)
- [Neural ODE Processes](https://mlsanity.com/paper/2103.12413/) (2021)
- [Heavy Ball Neural Ordinary Differential Equations](https://mlsanity.com/paper/2110.04840/) (2021)
- [Learning the Dynamics of Physical Systems from Sparse Observations with Finite Element Networks](https://mlsanity.com/paper/2203.08852/) (2022)
- [DiffEqFlux.jl - A Julia Library for Neural Differential Equations](https://mlsanity.com/paper/1902.02376/) (2019)
- [Opening the Blackbox: Accelerating Neural Differential Equations by Regularizing Internal Solver Heuristics](https://mlsanity.com/paper/2105.03918/) (2021)
- [ResNet After All? Neural ODEs and Their Numerical Solution](https://mlsanity.com/paper/2007.15386/) (2020)

## Built on by (most cited first, 10 of 10)

- [ODE-GS: Latent ODEs for Dynamic Scene Extrapolation with 3D Gaussian Splatting](https://mlsanity.com/paper/2506.05480/) (2025)
- [Identifiability Challenges in Sparse Linear Ordinary Differential Equations](https://mlsanity.com/paper/2506.09816/) (2025)
- [DeNOTS: Stable Deep Neural ODEs for Time Series](https://mlsanity.com/paper/2408.08055/) (2024)
- [Scientific Machine Learning of Flow Resistance Using Universal Shallow Water Equations with Differentiable Programming](https://mlsanity.com/paper/2502.12396/) (2025)
- [Benchmarking Continuous Time Models for Predicting Multiple Sclerosis Progression](https://mlsanity.com/paper/2302.07854/) (2023)
- [Generative Video Bi-flow](https://mlsanity.com/paper/2503.06364/) (2025)
- [A Generalizable Physics-guided Causal Model for Trajectory Prediction in Autonomous Driving](https://mlsanity.com/paper/2602.13936/) (2026)
- [SELDON: Supernova Explosions Learned by Deep ODE Networks](https://mlsanity.com/paper/2603.04392/) (2026)
- [evoxels: A differentiable physics framework for voxel-based microstructure simulations](https://mlsanity.com/paper/2507.21748/) (2025)
- [GRADSOLVE: fast exact gradients for ODE ensembles on GPUs](https://mlsanity.com/paper/2609.02876/) (2026)

## Abstract (arXiv)

We introduce an ODE solver for the PyTorch ecosystem that can solve multiple ODEs in parallel independently from each other while achieving significant performance gains. Our implementation tracks each ODE's progress separately and is carefully optimized for GPUs and compatibility with PyTorch's JIT compiler. Its design lets researchers easily augment any aspect of the solver and collect and analyze internal solver statistics. In our experiments, our implementation is up to 4.3 times faster per step than other ODE solvers and it is robust against within-batch interactions that lead other solvers to take up to 4 times as many steps. Code available at https://github.com/martenlienen/torchode