mlsanityData to 7 Oct 2026

Learning Differential Equations that are Easy to Solve

arXiv 2007.04504 ↗First posted 9 Jul 2020cs.LG stat.MLcited by 81

Not on the map. Why?

Abstract (arXiv)

Differential equations parameterized by neural networks become expensive to solve numerically as training progresses. We propose a remedy that encourages learned dynamics to be easier to solve. Specifically, we introduce a differentiable surrogate for the time cost of standard numerical solvers, using higher-order derivatives of solution trajectories. These derivatives are efficient to compute with Taylor-mode automatic differentiation. Optimizing this additional objective trades model performance against the time cost of solving the learned dynamics. We demonstrate our approach by training substantially faster, while nearly as accurate, models in supervised classification, density estimation, and time-series modelling tasks.

Key facts. Cited by 81 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 19; 2022: 22; 2023: 14; 2024: 11; 2025: 9; 2026: 6.

81 papers.

Sort:

By area:

  1. 2022Flow Matching for Generative Modelingcited by 5,426
  2. 2021GRAND: Graph Neural Diffusioncited by 258
  3. 2022GENIE: Higher-Order Denoising Diffusion Solverscited by 127
  4. 2021Neural Flows: Efficient Alternative to Neural ODEscited by 99
  5. 2023Minimizing Trajectory Curvature of ODE-based Generative Modelscited by 87
  6. 2021Beltrami Flow and Neural Diffusion on Graphscited by 84
  7. 2021Stabilizing Equilibrium Models by Jacobian Regularizationcited by 71
  8. 2022Deep Equilibrium Optical Flow Estimationcited by 63
  9. 2024Neural Flow Diffusion Models: Learnable Forward Process for Improved Diffusion Modellingcited by 43
  10. 2021Heavy Ball Neural Ordinary Differential Equationscited by 42
  11. 2021Neural Networks with Physics-Informed Architectures and Constraints for Dynamical Systems Modelingcited by 40
  12. 2021Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equationscited by 38
38 more
  1. 2021Constructing Neural Network-Based Models for Simulating Dynamical Systemscited by 29
  2. 2023Physics-Informed Machine Learning for Modeling and Control of Dynamical Systemscited by 28
  3. 2021Opening the Blackbox: Accelerating Neural Differential Equations by Regularizing Internal Solver Heuristicscited by 26
  4. 2023GECCO: Geometrically-Conditioned Point Diffusion Modelscited by 24
  5. 2024Scaling physics-informed hard constraints with mixture-of-expertscited by 24
  6. 2021Framing RNN as a kernel method: A neural ODE approachcited by 23
  7. 2022Neural Integral Equationscited by 23
  8. 2022Online Deep Equilibrium Learning for Regularization by Denoisingcited by 22
  9. 2023Generalization bounds for neural ordinary differential equations and deep residual networkscited by 21
  10. 2025Comprehensive Review of Neural Differential Equations for Time Series Analysiscited by 20
  11. 2025SDE Matching: Scalable and Simulation-Free Training of Latent Stochastic Differential Equationscited by 19
  12. 2021ACE-NODE: Attentive Co-Evolving Neural Ordinary Differential Equationscited by 15
  13. 2023Stabilized Neural Differential Equations for Learning Dynamics with Explicit Constraintscited by 15
  14. 2023Amortized Reparametrization: Efficient and Scalable Variational Inference for Latent SDEscited by 15
  15. 2022Homotopy-based training of NeuralODEs for accurate dynamics discoverycited by 14
  16. 2022Taylor-Lagrange Neural Ordinary Differential Equations: Toward Fast Training and Evaluation of Neural ODEscited by 13
  17. 2021Second-Order Neural ODE Optimizercited by 11
  18. 2024Nuclear Norm Regularization for Deep Learningcited by 11
  19. 2024Training Stiff Neural Ordinary Differential Equations with Implicit Single-Step Methodscited by 11
  20. 2021How Does Momentum Benefit Deep Neural Networks Architecture Design? A Few Case Studiescited by 10
  21. 2022Sparsity in Continuous-Depth Neural Networkscited by 10
  22. 2021Accelerating Neural ODEs Using Model Order Reductioncited by 9
  23. 2022Survival Mixture Density Networkscited by 9
  24. 2025Theoretical Guarantees for High Order Trajectory Refinement in Generative Flowscited by 9
  25. 2022TO-FLOW: Efficient Continuous Normalizing Flows with Temporal Optimization adjoint with Moving Speedcited by 8
  26. 2024Simulation-Free Training of Neural ODEs on Paired Datacited by 8
  27. 2022m^ast of two-dimensional electron gas: a neural canonical transformation studycited by 7
  28. 2022Proximal Implicit ODE Solvers for Accelerating Learning Neural ODEscited by 6
  29. 2022Conditional Permutation Invariant Flowscited by 6
  30. 2025Distinguishing Cause from Effect with Causal Velocity Modelscited by 6
  31. 2022Continuous Deep Equilibrium Models: Training Neural ODEs faster by integrating them to Infinitycited by 5
  32. 2022ICE-NODE: Integration of Clinical Embeddings with Neural Ordinary Differential Equationscited by 5
  33. 2024Rademacher Complexity of Neural ODEs via Chen-Fliess Seriescited by 5
  34. 2024Training Stiff Neural Ordinary Differential Equations with Explicit Exponential Integration Methodscited by 5
  35. 2025Continuous-Time Attention: PDE-Guided Mechanisms for Long-Sequence Transformerscited by 5
  36. 2021Multi-Resolution Continuous Normalizing Flowscited by 4
  37. 2022Neural Piecewise-Constant Delay Differential Equationscited by 4
  38. 2023Locally Regularized Neural Differential Equations: Some Black Boxes Were Meant to Remain Closed!cited by 4

50 most cited of 81 All 81 (JSON)