mlsanityData to 7 Oct 2026

Proximal Implicit ODE Solvers for Accelerating Learning Neural ODEs

arXiv 2204.08621 ↗First posted 19 Apr 2022math.NA cs.LG cs.NAcited by 6

Not on the map. Why?

Abstract (arXiv)

Learning neural ODEs often requires solving very stiff ODE systems, primarily using explicit adaptive step size ODE solvers. These solvers are computationally expensive, requiring the use of tiny step sizes for numerical stability and accuracy guarantees. This paper considers learning neural ODEs using implicit ODE solvers of different orders leveraging proximal operators. The proximal implicit solver consists of inner-outer iterations: the inner iterations approximate each implicit update step using a fast optimization algorithm, and the outer iterations solve the ODE system over time. The proximal implicit ODE solver guarantees superiority over explicit solvers in numerical stability and computational efficiency. We validate the advantages of proximal implicit solvers over existing popular neural ODE solvers on various challenging benchmark tasks, including learning continuous-depth graph neural networks and continuous normalizing flows.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 29 of its 64 references that are in the map. First posted on arXiv 19 Apr 2022.

Builds on

29 of 64 references.

References · 29

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Graph Attention Networkscited by 8,854
  3. 2018Neural Ordinary Differential Equationscited by 3,733
  4. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 487
  5. 2019Augmented Neural ODEscited by 396
  6. 2016Entropy-SGD: Biasing Gradient Descent Into Wide Valleyscited by 357
  7. 2021GRAND: Graph Neural Diffusioncited by 258
  8. 2020How to train your neural ODE: the world of Jacobian and kinetic regularizationcited by 228
  9. 2019Graph Neural Ordinary Differential Equationscited by 153
  10. 2020Dissecting Neural ODEscited by 150
  11. 2019Continuous Graph Neural Networkscited by 114
  12. 2019ANODE: Unconditionally Accurate Memory-Efficient Gradients for Neural ODEscited by 83
17 more
  1. 2020Learning Differential Equations that are Easy to Solvecited by 81
  2. 2020Adaptive Checkpoint Adjoint Method for Gradient Estimation in Neural ODEcited by 76
  3. 2020On Second Order Behaviour in Augmented Neural ODEscited by 68
  4. 2020STEER: Simple Temporal Regularization For Neural ODEscited by 58
  5. 2021Heavy Ball Neural Ordinary Differential Equationscited by 42
  6. 2020Hypersolvers: Toward Fast Continuous-Depth Modelscited by 33
  7. 2020ShapeFlow: Learnable Deformations Among 3D Shapescited by 28
  8. 2017Deep Relaxation: partial differential equations for optimizing deep neural networkscited by 27
  9. 2021Opening the Blackbox: Accelerating Neural Differential Equations by Regularizing Internal Solver Heuristicscited by 26
  10. 2015Proximal Algorithms in Statistics and Machine Learningcited by 18
  11. 2020Particle-based Energetic Variational Inferencecited by 14
  12. 2021Symplectic Adjoint Method for Exact Gradient of Neural ODE with Minimal Memorycited by 11
  13. 2020Sparsity Meets Robustness: Channel Pruning for the Feynman-Kac Formalism Principled Robust Deep Neural Netscited by 8
  14. 2020Stochastic Gradient Descent with Nonlinear Conjugate Gradient-Style Adaptive Momentumcited by 8
  15. 2019From differential equation solvers to accelerated first-order methods for convex optimizationcited by 5
  16. 2021Neural Ordinary Differential Equations for Data-Driven Reduced Order Modeling of Environmental Hydrodynamicscited by 5
  17. 2015Transformed Schatten-1 Iterative Thresholding Algorithms for Low Rank Matrix Completioncited by 3

Built on by

Cited by year

Citing papers per year: 2024: 3; 2025: 3.

6 papers.

  1. 2024Training Stiff Neural Ordinary Differential Equations with Implicit Single-Step Methodscited by 11
  2. 2024Training Stiff Neural Ordinary Differential Equations with Explicit Exponential Integration Methodscited by 5
  3. 2025The Vanishing Gradient Problem for Stiff Neural Differential Equationscited by 3
  4. 2024Semi-Implicit Neural Ordinary Differential Equationscited by 2
  5. 2025Physics-Informed Neural ODEs with Scale-Aware Residuals for Learning Stiff Biophysical Dynamicscited by 2
  6. 2025Stable CDE Autoencoders with Acuity Regularization for Offline Reinforcement Learning in Sepsis Treatmentcited by 0