# Distribution learning via neural differential equations: a nonparametric statistical perspective

arXiv: https://arxiv.org/abs/2309.01043 · HTML: https://mlsanity.com/paper/2309.01043/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 86 references that are in the map.
- First posted on arXiv 3 Sep 2023.

## Builds on (most cited first, 30 of 45)

- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [Stochastic Interpolants: A Unifying Framework for Flows and Diffusions](https://mlsanity.com/paper/2303.08797/) (2023)
- [Maximum Likelihood Training of Score-Based Diffusion Models](https://mlsanity.com/paper/2101.09258/) (2021)
- [FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models](https://mlsanity.com/paper/1810.01367/) (2018)
- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [The frontier of simulation-based inference](https://mlsanity.com/paper/1911.01429/) (2019)
- [Nonparametric regression using deep neural networks with ReLU activation function](https://mlsanity.com/paper/1708.06633/) (2017)
- [How to train your neural ODE: the world of Jacobian and kinetic regularization](https://mlsanity.com/paper/2002.02798/) (2020)
- [Neural Autoregressive Flows](https://mlsanity.com/paper/1804.00779/) (2018)
- [Conditional Image Generation with Score-Based Diffusion Models](https://mlsanity.com/paper/2111.13606/) (2021)
- [Optimal approximation of piecewise smooth functions using deep ReLU neural networks](https://mlsanity.com/paper/1709.05289/) (2017)
- [Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data](https://mlsanity.com/paper/2302.07194/) (2023)
- [Diffusion Models are Minimax Optimal Distribution Estimators](https://mlsanity.com/paper/2303.01861/) (2023)
- [Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionality](https://mlsanity.com/paper/1810.08033/) (2018)
- [The probability flow ODE is provably fast](https://mlsanity.com/paper/2305.11798/) (2023)
- [OT-Flow: Fast and Accurate Continuous Normalizing Flows via Optimal Transport](https://mlsanity.com/paper/2006.00104/) (2020)
- [Unconstrained Monotonic Neural Networks](https://mlsanity.com/paper/1908.05164/) (2019)
- [Convex Potential Flows: Universal Probability Distributions with Optimal Transport and Convex Optimization](https://mlsanity.com/paper/2012.05942/) (2020)
- [The phase diagram of approximation rates for deep neural networks](https://mlsanity.com/paper/1906.09477/) (2019)
- [ANODE: Unconditionally Accurate Memory-Efficient Gradients for Neural ODEs](https://mlsanity.com/paper/1902.10298/) (2019)

## Built on by (most cited first, 22 of 22)

- [Convergence of flow-based generative models via proximal gradient descent in Wasserstein space](https://mlsanity.com/paper/2310.17582/) (2023)
- [Convergence of Continuous Normalizing Flows for Learning Probability Distributions](https://mlsanity.com/paper/2404.00551/) (2024)
- [Efficient Neural Network Approaches for Conditional Optimal Transport with Applications in Bayesian Inference](https://mlsanity.com/paper/2310.16975/) (2023)
- [Gaussian Interpolation Flows](https://mlsanity.com/paper/2311.11475/) (2023)
- [On the minimax optimality of Flow Matching through the connection to kernel density estimation](https://mlsanity.com/paper/2504.13336/) (2025)
- [Flow Matching is Adaptive to Manifold Structures](https://mlsanity.com/paper/2602.22486/) (2026)
- [Kernel Approximation of Fisher-Rao Gradient Flows](https://mlsanity.com/paper/2410.20622/) (2024)
- [Local Flow Matching Generative Models](https://mlsanity.com/paper/2410.02548/) (2024)
- [Diffeomorphic Measure Matching with Kernels for Generative Modeling](https://mlsanity.com/paper/2402.08077/) (2024)
- [Robust Generative Learning with Lipschitz-Regularized α-Divergences Allows Minimal Assumptions on Target Distributions](https://mlsanity.com/paper/2405.13962/) (2024)
- [Constructive approximate transport maps with normalizing flows](https://mlsanity.com/paper/2412.19366/) (2024)
- [A friendly introduction to triangular transport](https://mlsanity.com/paper/2503.21673/) (2025)
- [Numerical and statistical analysis of NeuralODE with Runge-Kutta time integration](https://mlsanity.com/paper/2503.10729/) (2025)
- [Flow-based generative models as iterative algorithms in probability space](https://mlsanity.com/paper/2502.13394/) (2025)
- [Error Analysis of Bayesian Inverse Problems with Generative Priors](https://mlsanity.com/paper/2601.17374/) (2026)
- [One-Shot Generative Flows: Existence and Obstructions](https://mlsanity.com/paper/2604.15439/) (2026)
- [Continuous Temporal Learning of Probability Distributions via Neural ODEs with Applications in Continuous Glucose Monitoring Data](https://mlsanity.com/paper/2505.08698/) (2025)
- [Consistency of Learned Sparse Grid Quadrature Rules using NeuralODEs](https://mlsanity.com/paper/2507.01533/) (2025)
- [CroTad: A Contrastive Reinforcement Learning Framework for Online Trajectory Anomaly Detection](https://mlsanity.com/paper/2511.16929/) (2025)
- [Regularity of Solutions to Beckmann's Parametric Optimal Transport](https://mlsanity.com/paper/2603.19755/) (2026)
- [Generative Adversarial Learning from Deterministic Processes](https://mlsanity.com/paper/2605.18425/) (2026)
- [Constructive interpolation and generalization rates for neural ODEs: a control perspective](https://mlsanity.com/paper/2606.00469/) (2026)

## Abstract (arXiv)

Ordinary differential equations (ODEs), via their induced flow maps, provide a powerful framework to parameterize invertible transformations for the purpose of representing complex probability distributions. While such models have achieved enormous success in machine learning, particularly for generative modeling and density estimation, little is known about their statistical properties. This work establishes the first general nonparametric statistical convergence analysis for distribution learning via ODE models trained through likelihood maximization. We first prove a convergence theorem applicable to arbitrary velocity field classes F satisfying certain simple boundary constraints. This general result captures the trade-off between approximation error (`bias') and the complexity of the ODE model (`variance'). We show that the latter can be quantified via the C1-metric entropy of the class mathcal F. We then apply this general framework to the setting of Ck-smooth target densities, and establish nearly minimax-optimal convergence rates for two relevant velocity field classes mathcal F: Ck functions and neural networks. The latter is the practically important case of neural ODEs. Our proof techniques require a careful synthesis of (i) analytical stability results for ODEs, (ii) classical theory for sieved M-estimators, and (iii) recent results on approximation rates and metric entropies of neural network classes. The results also provide theoretical insight on how the choice of velocity field class, and the dependence of this choice on sample size n (e.g., the scaling of width, depth, and sparsity of neural network classes), impacts statistical performance.