# Differentiable Programming for Earth System Modeling

arXiv: https://arxiv.org/abs/2208.13825 · HTML: https://mlsanity.com/paper/2208.13825/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 107 references that are in the map.
- First posted on arXiv 29 Aug 2022.

## Builds on (most cited first, 24 of 24)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [On Lazy Training in Differentiable Programming](https://mlsanity.com/paper/1812.07956/) (2018)
- [Machine learning accelerated computational fluid dynamics](https://mlsanity.com/paper/2102.01010/) (2021)
- [Universal Differential Equations for Scientific Machine Learning](https://mlsanity.com/paper/2001.04385/) (2020)
- [Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers](https://mlsanity.com/paper/2007.00016/) (2020)
- [Efficient and Modular Implicit Differentiation](https://mlsanity.com/paper/2105.15183/) (2021)
- [Deep learning to represent sub-grid processes in climate models](https://mlsanity.com/paper/1806.04731/) (2018)
- [Learning to Control PDEs with Differentiable Physics](https://mlsanity.com/paper/2001.07457/) (2020)
- [Deep Learning for Physical Processes: Incorporating Prior Scientific Knowledge](https://mlsanity.com/paper/1711.07970/) (2017)
- [Gradients are Not All You Need](https://mlsanity.com/paper/2111.05803/) (2021)
- [A Differentiable Programming System to Bridge Machine Learning and Scientific Computing](https://mlsanity.com/paper/1907.07587/) (2019)
- [Stiff Neural Ordinary Differential Equations](https://mlsanity.com/paper/2103.15341/) (2021)
- [Earth System Modeling 2.0: A Blueprint for Models That Learn From Observations and Targeted High-Resolution Simulations](https://mlsanity.com/paper/1709.00037/) (2017)
- [Will Artificial Intelligence supersede Earth System and Climate Models?](https://mlsanity.com/paper/2101.09126/) (2021)
- [JAX-FLUIDS: A fully-differentiable high-order computational fluid dynamics solver for compressible two-phase flows](https://mlsanity.com/paper/2203.13760/) (2022)
- [A posteriori learning for quasi-geostrophic turbulence parametrization](https://mlsanity.com/paper/2204.03911/) (2022)
- [A computational framework for infinite-dimensional Bayesian inverse problems. Part I: The linearized case, with application to global seismic inversion](https://mlsanity.com/paper/1308.1313/) (2013)
- [Achieving Conservation of Energy in Neural Network Emulators for Climate Modeling](https://mlsanity.com/paper/1906.06622/) (2019)
- [Instead of Rewriting Foreign Code for Machine Learning, Automatically Synthesize Fast Gradients](https://mlsanity.com/paper/2010.01709/) (2020)
- [From calibration to parameter learning: Harnessing the scaling effects of big data in geoscientific modeling](https://mlsanity.com/paper/2007.15751/) (2020)
- [A computational framework for infinite-dimensional Bayesian inverse problems: Part II. Stochastic Newton MCMC with application to ice sheet flow inverse problems](https://mlsanity.com/paper/1308.6221/) (2013)
- [hIPPYlib: An Extensible Software Framework for Large-Scale Inverse Problems Governed by PDEs; Part I: Deterministic Inversion and Linearized Bayesian Inference](https://mlsanity.com/paper/1909.03948/) (2019)
- [Sensitivity analysis on chaotic dynamical system by Non-Intrusive Least Square Shadowing (NILSS)](https://mlsanity.com/paper/1611.00880/) (2016)

## Built on by (most cited first, 22 of 22)

- [Neural General Circulation Models for Weather and Climate](https://mlsanity.com/paper/2311.07222/) (2023)
- [Stabilized Neural Differential Equations for Learning Dynamics with Explicit Constraints](https://mlsanity.com/paper/2306.09739/) (2023)
- [Time Series Predictions in Unmonitored Sites: A Survey of Machine Learning Techniques in Water Resources](https://mlsanity.com/paper/2308.09766/) (2023)
- [Gradient-free online learning of subgrid-scale dynamics with neural emulators](https://mlsanity.com/paper/2310.19385/) (2023)
- [Projected Neural Differential Equations for Learning Constrained Dynamics](https://mlsanity.com/paper/2410.23667/) (2024)
- [JAX-MPM: A Learning-Augmented Differentiable Meshfree Framework for GPU-Accelerated Lagrangian Simulation and Geophysical Inverse Modeling](https://mlsanity.com/paper/2507.04192/) (2025)
- [Reconstructing Historical Climate Fields With Deep Learning](https://mlsanity.com/paper/2311.18348/) (2023)
- [Augmented Invertible Koopman Autoencoder for long-term time series forecasting](https://mlsanity.com/paper/2503.12930/) (2025)
- [Constructing Extreme Heatwave Storylines with Differentiable Climate Models](https://mlsanity.com/paper/2506.10660/) (2025)
- [Extreme Event Prediction with Multi-agent Reinforcement Learning-based Parametrization of Atmospheric and Oceanic Turbulence](https://mlsanity.com/paper/2312.00907/) (2023)
- [Towards fully differentiable neural ocean model with Veros](https://mlsanity.com/paper/2511.17427/) (2025)
- [Differentiable Inverse Modeling with Physics-Constrained Latent Diffusion for Heterogeneous Subsurface Parameter Fields](https://mlsanity.com/paper/2512.22421/) (2025)
- [Systematic LLM Translation of Legacy Scientific Code to Differentiable Frameworks: Application to a Land Surface Model](https://mlsanity.com/paper/2606.07681/) (2026)
- [Adjoint-based online learning of two-layer quasi-geostrophic baroclinic turbulence](https://mlsanity.com/paper/2411.14106/) (2024)
- [Learning vertical coordinates via automatic differentiation of a dynamical core](https://mlsanity.com/paper/2512.17877/) (2025)
- [Learning to Optimize by Differentiable Programming](https://mlsanity.com/paper/2601.16510/) (2026)
- [ADDA: a Modular Framework for Representing, Simulating and Assimilating Dynamics with End-to-end Differentiability](https://mlsanity.com/paper/2608.23297/) (2026)
- [Emulating Non-Differentiable Metrics via Knowledge-Guided Learning: Introducing the Minkowski Image Loss](https://mlsanity.com/paper/2604.11422/) (2026)
- [Emulating the Forced Response of Climate Models with Generative Machine Learning](https://mlsanity.com/paper/2605.16929/) (2026)
- [Optimizing Geoengineering Interventions Using Differentiable Climate Models](https://mlsanity.com/paper/2609.12528/) (2026)
- [Variational Augmented Invertible Koopman Autoencoder for probabilistic time series forecasting](https://mlsanity.com/paper/2609.37435/) (2026)
- [AEGIS: Differentiable Mars Climate Model with Neural Closures](https://mlsanity.com/paper/2610.04081/) (2026)

## Abstract (arXiv)

Earth System Models (ESMs) are the primary tools for investigating future Earth system states at time scales from decades to centuries, especially in response to anthropogenic greenhouse gas release. State-of-the-art ESMs can reproduce the observational global mean temperature anomalies of the last 150 years. Nevertheless, ESMs need further improvements, most importantly regarding (i) the large spread in their estimates of climate sensitivity, i.e., the temperature response to increases in atmospheric greenhouse gases, (ii) the modeled spatial patterns of key variables such as temperature and precipitation, (iii) their representation of extreme weather events, and (iv) their representation of multistable Earth system components and their ability to predict associated abrupt transitions. Here, we argue that making ESMs automatically differentiable has huge potential to advance ESMs, especially with respect to these key shortcomings. First, automatic differentiability would allow objective calibration of ESMs, i.e., the selection of optimal values with respect to a cost function for a large number of free parameters, which are currently tuned mostly manually. Second, recent advances in Machine Learning (ML) and in the amount, accuracy, and resolution of observational data promise to be helpful with at least some of the above aspects because ML may be used to incorporate additional information from observations into ESMs. Automatic differentiability is an essential ingredient in the construction of such hybrid models, combining process-based ESMs with ML components. We document recent work showcasing the potential of automatic differentiation for a new generation of substantially improved, data-informed ESMs.