# Differentiable Programming for Differential Equations: A Review

arXiv: https://arxiv.org/abs/2406.09699 · HTML: https://mlsanity.com/paper/2406.09699/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jun 2024.

## Built on by (most cited first, 21 of 21)

- [What You See is Not What You Get: Neural Partial Differential Equations and The Illusion of Learning](https://mlsanity.com/paper/2411.15101/) (2024)
- [JAX-MPM: A Learning-Augmented Differentiable Meshfree Framework for GPU-Accelerated Lagrangian Simulation and Geophysical Inverse Modeling](https://mlsanity.com/paper/2507.04192/) (2025)
- [Semi-Explicit Neural DAEs: Learning Long-Horizon Dynamical Systems with Algebraic Constraints](https://mlsanity.com/paper/2505.20515/) (2025)
- [Scientific Machine Learning of Flow Resistance Using Universal Shallow Water Equations with Differentiable Programming](https://mlsanity.com/paper/2502.12396/) (2025)
- [Towards fully differentiable neural ocean model with Veros](https://mlsanity.com/paper/2511.17427/) (2025)
- [Adjoint-based online learning of two-layer quasi-geostrophic baroclinic turbulence](https://mlsanity.com/paper/2411.14106/) (2024)
- [Learning vertical coordinates via automatic differentiation of a dynamical core](https://mlsanity.com/paper/2512.17877/) (2025)
- [Learning to Optimize by Differentiable Programming](https://mlsanity.com/paper/2601.16510/) (2026)
- [ADDA: a Modular Framework for Representing, Simulating and Assimilating Dynamics with End-to-end Differentiability](https://mlsanity.com/paper/2608.23297/) (2026)
- [Finding the Underlying Viscoelastic Constitutive Equation via Universal Differential Equations and Differentiable Physics](https://mlsanity.com/paper/2501.00556/) (2024)
- [MixFunn: A Neural Network for Differential Equations with Improved Generalization and Interpretability](https://mlsanity.com/paper/2503.22528/) (2025)
- [A Common Interface for Automatic Differentiation](https://mlsanity.com/paper/2505.05542/) (2025)
- [Inferring processes within dynamic forest models using hybrid modeling](https://mlsanity.com/paper/2508.01228/) (2025)
- [CODE: A global approach to ODE dynamics learning](https://mlsanity.com/paper/2511.15619/) (2025)
- [An adjoint method for training data-driven reduced-order models](https://mlsanity.com/paper/2601.07579/) (2026)
- [A solver-in-the-loop framework for end-to-end differentiable coastal hydrodynamics](https://mlsanity.com/paper/2604.07129/) (2026)
- [Performance and Explainability Requirements of Evolutionary Algorithms in Real-World Physics-Informed Optimization](https://mlsanity.com/paper/2605.28164/) (2026)
- [Physics-Aware Linearized ADMM and Its Unrolling](https://mlsanity.com/paper/2606.01652/) (2026)
- [Integrating Mechanistic and Data-Driven Models for Neurological Disorders through Differentiable Programming](https://mlsanity.com/paper/2606.06094/) (2026)
- [Helix-FNO: Spectral-Domain Operator Learning Coupled with a High-Fidelity Mechanistic Model for Fast Surrogate Simulation](https://mlsanity.com/paper/2609.22129/) (2026)
- [Variational Augmented Invertible Koopman Autoencoder for probabilistic time series forecasting](https://mlsanity.com/paper/2609.37435/) (2026)

## Abstract (arXiv)

The differentiable programming paradigm is a cornerstone of modern scientific computing. It refers to numerical methods for computing the gradient of a numerical model's output. Many scientific models are based on differential equations, where differentiable programming plays a crucial role in calculating model sensitivities, inverting model parameters, and training hybrid models that combine differential equations with data-driven approaches. Furthermore, recognizing the strong synergies between inverse methods and machine learning offers the opportunity to establish a coherent framework applicable to both fields. Differentiating functions based on the numerical solution of differential equations is non-trivial. Numerous methods based on a wide variety of paradigms have been proposed in the literature, each with pros and cons specific to the type of problem investigated. Here, we provide a comprehensive review of existing techniques to compute derivatives of numerical solutions of differential equations. We first discuss the importance of gradients of solutions of differential equations in a variety of scientific domains. Second, we lay out the mathematical foundations of the various approaches and compare them with each other. Third, we cover the computational considerations and explore the solutions available in modern scientific software. Last but not least, we provide best-practices and recommendations for practitioners. We hope that this work accelerates the fusion of scientific models and data, and fosters a modern approach to scientific modelling.