# 4D-Var using Hessian approximation and backpropagation applied to automatically-differentiable numerical and machine learning models

arXiv: https://arxiv.org/abs/2408.02767 · HTML: https://mlsanity.com/paper/2408.02767/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 79 references that are in the map.
- First posted on arXiv 5 Aug 2024.

## Builds on (most cited first, 15 of 15)

- [Tune: A Research Platform for Distributed Model Selection and Training](https://mlsanity.com/paper/1807.05118/) (2018)
- [The CMA Evolution Strategy: A Tutorial](https://mlsanity.com/paper/1604.00772/) (2016)
- [WeatherBench 2: A benchmark for the next generation of data-driven global weather models](https://mlsanity.com/paper/2308.15560/) (2023)
- [Spherical Fourier Neural Operators: Learning Stable Dynamics on the Sphere](https://mlsanity.com/paper/2306.03838/) (2023)
- [FourCastNet: Accelerating Global High-Resolution Weather Forecasting using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2208.05419/) (2022)
- [Backpropagation Algorithms and Reservoir Computing in Recurrent Neural Networks for the Forecasting of Complex Spatiotemporal Dynamics](https://mlsanity.com/paper/1910.05266/) (2019)
- [Machine learning with data assimilation and uncertainty quantification for dynamical systems: a review](https://mlsanity.com/paper/2303.10462/) (2023)
- [A Systematic Exploration of Reservoir Computing for Forecasting Complex Spatiotemporal Dynamics](https://mlsanity.com/paper/2201.08910/) (2022)
- [FengWu-4DVar: Coupling the Data-driven Weather Forecasting Model with 4D Variational Assimilation](https://mlsanity.com/paper/2312.12455/) (2023)
- [Integrating Recurrent Neural Networks with Data Assimilation for Scalable Data-Driven State Estimation](https://mlsanity.com/paper/2109.12269/) (2021)
- [Data driven weather forecasts trained and initialised directly from observations](https://mlsanity.com/paper/2407.15586/) (2024)
- [Data Assimilation with Machine Learning Surrogate Models: A Case Study with FourCastNet](https://mlsanity.com/paper/2405.13180/) (2024)
- [A multi-model ensemble Kalman filter for data assimilation and forecasting](https://mlsanity.com/paper/2202.02272/) (2022)
- [Oceananigans.jl: A Julia library that achieves breakthrough resolution, memory and energy efficiency in global ocean simulations](https://mlsanity.com/paper/2309.06662/) (2023)
- [Convergent least-squares optimisation methods for variational data assimilation](https://mlsanity.com/paper/2107.12361/) (2021)

## Built on by (most cited first, 6 of 6)

- [Constructing Extreme Heatwave Storylines with Differentiable Climate Models](https://mlsanity.com/paper/2506.10660/) (2025)
- [ADDA: a Modular Framework for Representing, Simulating and Assimilating Dynamics with End-to-end Differentiability](https://mlsanity.com/paper/2608.23297/) (2026)
- [PnP-DA: Towards Principled Plug-and-Play Integration of Variational Data Assimilation and Generative Models](https://mlsanity.com/paper/2508.00325/) (2025)
- [Meta-Learning Fourier Neural Operators for Hessian Inversion and Enhanced Variational Data Assimilation](https://mlsanity.com/paper/2509.22949/) (2025)
- [DAWIS: Data Assimilation with Windowed Inverse Sampling via Multitask Interpolants](https://mlsanity.com/paper/2610.03314/) (2026)
- [Sensitivity Fidelity Reveals Physical Dependencies Beyond Forecast Accuracy in Deep-Learning Earth-System Models](https://mlsanity.com/paper/2610.15540/) (2026)

## Abstract (arXiv)

Constraining a numerical weather prediction (NWP) model with observations via 4D variational (4D-Var) data assimilation is often difficult to implement in practice due to the need to develop and maintain a software-based tangent linear model and adjoint model. One of the most common 4D-Var algorithms uses an incremental update procedure, which has been shown to be an approximation of the Gauss-Newton method. Here we demonstrate that when using a forecast model that supports automatic differentiation, an efficient and in some cases more accurate alternative approximation of the Gauss-Newton method can be applied by combining backpropagation of errors with Hessian approximation. This approach can be used with either a conventional numerical model implemented within a software framework that supports automatic differentiation, or a machine learning (ML) based surrogate model. We test the new approach on a variety of Lorenz-96 and quasi-geostrophic models. The results indicate potential for a deeper integration of modeling, data assimilation, and new technologies in a next-generation of operational forecast systems that leverage weather models designed to support automatic differentiation.