# TorchDyn: A Neural Differential Equations Library

arXiv: https://arxiv.org/abs/2009.09346 · HTML: https://mlsanity.com/paper/2009.09346/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Sep 2020.

## Built on by (most cited first, 17 of 17)

- [Deep networks for system identification: a Survey](https://mlsanity.com/paper/2301.12832/) (2023)
- [Physics-Guided, Physics-Informed, and Physics-Encoded Neural Networks in Scientific Computing](https://mlsanity.com/paper/2211.07377/) (2022)
- [TorchDEQ: A Library for Deep Equilibrium Models](https://mlsanity.com/paper/2310.18605/) (2023)
- [Differentiable Multiple Shooting Layers](https://mlsanity.com/paper/2106.03885/) (2021)
- [Neural Ordinary Differential Equations for Nonlinear System Identification](https://mlsanity.com/paper/2203.00120/) (2022)
- [Sparse Flows: Pruning Continuous-depth Models](https://mlsanity.com/paper/2106.12718/) (2021)
- [Accelerating Neural ODEs Using Model Order Reduction](https://mlsanity.com/paper/2105.14070/) (2021)
- [Continuous-Depth Neural Models for Dynamic Graph Prediction](https://mlsanity.com/paper/2106.11581/) (2021)
- [Self-Similarity Priors: Neural Collages as Differentiable Fractal Representations](https://mlsanity.com/paper/2204.07673/) (2022)
- [Optimal Energy Shaping via Neural Approximators](https://mlsanity.com/paper/2101.05537/) (2021)
- [Neural Solvers for Fast and Accurate Numerical Optimal Control](https://mlsanity.com/paper/2203.08072/) (2022)
- [A Neural Ordinary Differential Equation Model for Visualizing Deep Neural Network Behaviors in Multi-Parametric MRI based Glioma Segmentation](https://mlsanity.com/paper/2203.00628/) (2022)
- [Neural Port-Hamiltonian Models for Nonlinear Distributed Control: An Unconstrained Parametrization Approach](https://mlsanity.com/paper/2411.10096/) (2024)
- [High-order expansion of Neural Ordinary Differential Equations flows](https://mlsanity.com/paper/2504.08769/) (2025)
- [Freudian and Newtonian Recurrent Cell for Sequential Recommendation](https://mlsanity.com/paper/2102.07645/) (2021)
- [On the Benefits of Biophysical Synapses](https://mlsanity.com/paper/2303.04944/) (2023)
- [Deterministic Prediction as Global Velocity: Flow Matching for Probabilistic Time-Series Forecast](https://mlsanity.com/paper/2610.18621/) (2026)

## Abstract (arXiv)

Continuous-depth learning has recently emerged as a novel perspective on deep learning, improving performance in tasks related to dynamical systems and density estimation. Core to these approaches is the neural differential equation, whose forward passes are the solutions of an initial value problem parametrized by a neural network. Unlocking the full potential of continuous-depth models requires a different set of software tools, due to peculiar differences compared to standard discrete neural networks, e.g inference must be carried out via numerical solvers. We introduce TorchDyn, a PyTorch library dedicated to continuous-depth learning, designed to elevate neural differential equations to be as accessible as regular plug-and-play deep learning primitives. This objective is achieved by identifying and subdividing different variants into common essential components, which can be combined and freely repurposed to obtain complex compositional architectures. TorchDyn further offers step-by-step tutorials and benchmarks designed to guide researchers and contributors.