mlsanityData to 14 Oct 2026

Efficient, Accurate and Stable Gradients for Neural ODEs

arXiv 2410.11648 ↗First posted 15 Oct 2024cs.LG stat.MLcited by 9

Not on the map. Why?

Abstract (arXiv)

Training Neural ODEs requires backpropagating through an ODE solve. The state-of-the-art backpropagation method is recursive checkpointing that balances recomputation with memory cost. Here, we introduce a class of algebraically reversible ODE solvers that significantly improve upon both the time and memory cost of recursive checkpointing. The reversible solvers presented calculate exact gradients, are high-order and numerically stable -- strictly improving on previous reversible architectures.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 22 of its 41 references that are in the map. First posted on arXiv 15 Oct 2024.

Builds on

22 of 41 references.

References · 22

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018Neural Ordinary Differential Equationscited by 3,779
  3. 2015Automatic differentiation in machine learning: a surveycited by 855
  4. 2019Hamiltonian Neural Networkscited by 712
  5. 2019Deep Equilibrium Modelscited by 615
  6. 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 594
  7. 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 506
  8. 2020Lagrangian Neural Networkscited by 398
  9. 2020Universal Differential Equations for Scientific Machine Learningcited by 353
  10. 2018The UEA multivariate time series classification archive, 2018cited by 338
  11. 2020Scalable Gradients for Stochastic Differential Equationscited by 271
  12. 2017The Reversible Residual Network: Backpropagation Without Storing Activationscited by 217
10 more
  1. 2021Neural SDEs as Infinite-Dimensional GANscited by 161
  2. 2019ANODE: Unconditionally Accurate Memory-Efficient Gradients for Neural ODEscited by 84
  3. 2021Efficient and Accurate Gradients for Neural SDEscited by 71
  4. 2021Neural Controlled Differential Equations for Online Prediction Taskscited by 46
  5. 2020Parameterized Neural Ordinary Differential Equations: Applications to Computational Physics Problemscited by 45
  6. 2021MALI: A memory efficient and reverse accurate integrator for Neural ODEscited by 42
  7. 2021Momentum Residual Neural Networkscited by 37
  8. 2023Non-adversarial training of Neural SDEs with signature kernel scorescited by 37
  9. 2021Forecasting the outcome of spintronic experiments with Neural Ordinary Differential Equationscited by 13
  10. 2023Neural Ideal Large Eddy Simulation: Modeling Turbulence with Neural Stochastic Differential Equationscited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 1.

9 papers.

  1. 2025Neural ODE Transformers: Analyzing Internal Dynamics and Adaptive Fine-tuningcited by 13
  2. 2025ODE-GS: Latent ODEs for Dynamic Scene Extrapolation with 3D Gaussian Splattingcited by 11
  3. 2025Reversible Deep Equilibrium Modelscited by 5
  4. 2025Greed is Good: A Unifying Perspective on Guided Generationcited by 2
  5. 2025Deep Neural Networks Inspired by Differential Equationscited by 2
  6. 2025Contrastive Diffusion Alignment: Learning Structured Latents for Controllable Generationcited by 2
  7. 2025Efficient Neural SDE Training using Wiener-Space Cubaturecited by 1
  8. 2025Explicit and Effectively Symmetric Schemes for Neural SDEs on Lie Groupscited by 1
  9. 2026Stable and Near-Reversible Diffusion ODE Solvers for Image Editingcited by 0