mlsanityData to 7 Oct 2026

How to train your neural ODE: the world of Jacobian and kinetic regularization

arXiv 2002.02798 ↗First posted 7 Feb 2020stat.ML cs.LGcited by 228

Not on the map. Why?

Cited by 35 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Training neural ODEs on large datasets has not been tractable due to the necessity of allowing the adaptive numerical ODE solver to refine its step size to very small values. In practice this leads to dynamics equivalent to many hundreds or even thousands of layers. In this paper, we overcome this apparent difficulty by introducing a theoretically-grounded combination of both optimal transport and stability regularizations which encourage neural ODEs to prefer simpler dynamics out of all the dynamics that solve a problem well. Simpler dynamics lead to faster convergence and to fewer discretizations of the solver, considerably decreasing wall-clock time without loss in performance. Our approach allows us to train neural ODE-based generative models to the same performance as the unregularized dynamics, with significant reductions in training time. This brings neural ODEs closer to practical relevance in large-scale applications.

Key facts. Cited by 228 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 34; 2022: 45; 2023: 46; 2024: 41; 2025: 32; 2026: 30.

228 papers.

Sort:

By area:

  1. 2022Flow Matching for Generative Modelingcited by 5,426
  2. 2022Building Normalizing Flows with Stochastic Interpolantscited by 964
  3. 2023Stochastic Interpolants: A Unifying Framework for Flows and Diffusionscited by 897
  4. 2023Improving and generalizing flow-based generative models with minibatch optimal transportcited by 835
  5. 2023Multisample Flow Matching: Straightening Flows with Minibatch Couplingscited by 267
  6. 2021GRAND: Graph Neural Diffusioncited by 258
  7. 2021Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Modelscited by 251
  8. 2021E(n) Equivariant Normalizing Flowscited by 165
  9. 2023SE(3)-Stochastic Flow Matching for Protein Backbone Generationcited by 147
  10. 2023Equivariant flow matchingcited by 131
  11. 2022GENIE: Higher-Order Denoising Diffusion Solverscited by 127
  12. 2021An Introduction to Deep Generative Modelingcited by 104
38 more
  1. 2021Neural Flows: Efficient Alternative to Neural ODEscited by 99
  2. 2024PeRFlow: Piecewise Rectified Flow as Universal Plug-and-Play Acceleratorcited by 95
  3. 2022Action Matching: Learning Stochastic Dynamics from Samplescited by 94
  4. 2023Minimizing Trajectory Curvature of ODE-based Generative Modelscited by 87
  5. 2022STaSy: Score-based Tabular data Synthesiscited by 85
  6. 2021JFB: Jacobian-Free Backpropagation for Implicit Networkscited by 84
  7. 2021Beltrami Flow and Neural Diffusion on Graphscited by 84
  8. 2023Simulation-free Schr"odinger bridges via score and flow matchingcited by 81
  9. 2022Maximum Likelihood Training for Score-Based Diffusion ODEs by High-Order Denoising Score Matchingcited by 79
  10. 2023CoDi: Co-evolving Contrastive Diffusion Models for Mixed-type Tabular Synthesiscited by 78
  11. 2024Metric Flow Matching for Smooth Interpolations on the Data Manifoldcited by 74
  12. 2021Stabilizing Equilibrium Models by Jacobian Regularizationcited by 71
  13. 2022Neural Conservation Laws: A Divergence-Free Perspectivecited by 67
  14. 2023Recent Advances in Optimal Transport for Machine Learningcited by 66
  15. 2021Closed-form Continuous-time Neural Modelscited by 65
  16. 2022Manifold Interpolating Optimal-Transport Flows for Trajectory Inferencecited by 63
  17. 2023Improved Techniques for Maximum Likelihood Estimation for Diffusion ODEscited by 54
  18. 2022Sequential Neural Score Estimation: Likelihood-Free Inference with Conditional Score Based Diffusion Modelscited by 50
  19. 2021MALI: A memory efficient and reverse accurate integrator for Neural ODEscited by 42
  20. 2021Heavy Ball Neural Ordinary Differential Equationscited by 42
  21. 2021Neural Point Process for Learning Spatiotemporal Event Dynamicscited by 42
  22. 2023On Kinetic Optimal Probability Paths for Generative Modelscited by 42
  23. 2023Generalized Schr"odinger Bridge Matchingcited by 40
  24. 2022Normalizing flow neural networks by JKO schemecited by 39
  25. 2023MissDiff: Training Diffusion Models on Tabular Data with Missing Valuescited by 38
  26. 2023Convergence of flow-based generative models via proximal gradient descent in Wasserstein spacecited by 34
  27. 2021Constructing Neural Network-Based Models for Simulating Dynamical Systemscited by 29
  28. 2021Implicit Normalizing Flowscited by 28
  29. 2023Physics-Informed Machine Learning for Modeling and Control of Dynamical Systemscited by 28
  30. 2021OCT-GAN: Neural ODE-based Conditional Tabular GANscited by 27
  31. 2021Opening the Blackbox: Accelerating Neural Differential Equations by Regularizing Internal Solver Heuristicscited by 26
  32. 2022Efficient-VDVAE: Less is morecited by 26
  33. 2024Dynamic Conditional Optimal Transport through Simulation-Free Flowscited by 24
  34. 2022Neural Integral Equationscited by 23
  35. 2022EXIT: Extrapolation and Interpolation-based Neural Controlled Differential Equations for Time-series Classification and Forecastingcited by 22
  36. 2023Distribution learning via neural differential equations: a nonparametric statistical perspectivecited by 22
  37. 2024Learning diffusion at lightspeedcited by 21
  38. 2023Algorithms for mean-field variational inference via polyhedral optimization in the Wasserstein spacecited by 20

50 most cited of 228 All 228 (JSON)