# Online model error correction with neural networks in the incremental 4D-Var framework

arXiv: https://arxiv.org/abs/2210.13817 · HTML: https://mlsanity.com/paper/2210.13817/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 64 references that are in the map.
- First posted on arXiv 25 Oct 2022.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Deep learning to represent sub-grid processes in climate models](https://mlsanity.com/paper/1806.04731/) (2018)
- [Combining data assimilation and machine learning to emulate a dynamical model from sparse and noisy observations: a case study with the Lorenz 96 model](https://mlsanity.com/paper/2001.01520/) (2020)
- [Hybrid Forecasting of Chaotic Processes: Using Machine Learning in Conjunction with a Knowledge-Based Model](https://mlsanity.com/paper/1803.04779/) (2018)
- [Physics Guided RNNs for Modeling Dynamical Systems: A Case Study in Simulating Lake Temperature Profiles](https://mlsanity.com/paper/1810.13075/) (2018)
- [Machine Learning for Stochastic Parameterization: Generative Adversarial Networks in the Lorenz '96 Model](https://mlsanity.com/paper/1909.04711/) (2019)
- [Using machine learning to correct model error in data assimilation and forecast applications](https://mlsanity.com/paper/2010.12605/) (2020)
- [Combining data assimilation and machine learning to infer unresolved scale parametrisation](https://mlsanity.com/paper/2009.04318/) (2020)
- [Bayesian inference of chaotic dynamics by merging data assimilation, machine learning and expectation-maximization](https://mlsanity.com/paper/2001.06270/) (2020)
- [A Framework for Machine Learning of Model Error in Dynamical Systems](https://mlsanity.com/paper/2107.06658/) (2021)
- [Machine Learning for Prediction with Missing Dynamics](https://mlsanity.com/paper/1910.05861/) (2019)
- [Supervised learning from noisy observations: Combining machine-learning techniques with data assimilation](https://mlsanity.com/paper/2007.07383/) (2020)
- [Combining Machine Learning with Knowledge-Based Modeling for Scalable Forecasting and Subgrid-Scale Closure of Large, Complex, Spatiotemporal Systems](https://mlsanity.com/paper/2002.05514/) (2020)
- [A Fortran-Keras Deep Learning Bridge for Scientific Computing](https://mlsanity.com/paper/2004.10652/) (2020)
- [Applying machine learning to improve simulations of a chaotic dynamical system using empirical error correction](https://mlsanity.com/paper/1904.10904/) (2019)
- [Online learning of both state and dynamics using ensemble Kalman filters](https://mlsanity.com/paper/2006.03859/) (2020)
- [Machine Learning, Deepest Learning: Statistical Data Assimilation Problems](https://mlsanity.com/paper/1707.01415/) (2017)
- [A comparison of combined data assimilation and machine learning methods for offline and online model error correction](https://mlsanity.com/paper/2107.11114/) (2021)
- [State, global and local parameter estimation using local ensemble Kalman filters: applications to online machine learning of chaotic dynamics](https://mlsanity.com/paper/2107.11253/) (2021)
- [Discovery of interpretable structural model errors by combining Bayesian sparse regression and data assimilation: A chaotic Kuramoto-Sivashinsky test case](https://mlsanity.com/paper/2110.00546/) (2021)

## Built on by (most cited first, 14 of 14)

- [Machine learning with data assimilation and uncertainty quantification for dynamical systems: a review](https://mlsanity.com/paper/2303.10462/) (2023)
- [Deep Generative Data Assimilation in Multimodal Setting](https://mlsanity.com/paper/2404.06665/) (2024)
- [AI Foundation Models for Weather and Climate: Applications, Design, and Implementation](https://mlsanity.com/paper/2309.10808/) (2023)
- [CGKN: A Deep Learning Framework for Modeling Complex Dynamical Systems and Efficient Data Assimilation](https://mlsanity.com/paper/2410.20072/) (2024)
- [CGNSDE: Conditional Gaussian Neural Stochastic Differential Equation for Modeling Complex Systems and Data Assimilation](https://mlsanity.com/paper/2404.06749/) (2024)
- [Joint Parameter and Parameterization Inference with Uncertainty Quantification through Differentiable Programming](https://mlsanity.com/paper/2403.02215/) (2024)
- [TorchDA: A Python package for performing data assimilation with deep learning forward and transformation functions](https://mlsanity.com/paper/2409.00244/) (2024)
- [Ensemble Kalman filter in latent space using a variational autoencoder pair](https://mlsanity.com/paper/2502.12987/) (2025)
- [Online Calibration of Deep Learning Sub-Models for Hybrid Numerical Modeling Systems](https://mlsanity.com/paper/2311.10665/) (2023)
- [Online machine-learning forecast uncertainty estimation for sequential data assimilation](https://mlsanity.com/paper/2305.08874/) (2023)
- [Development of an offline and online hybrid model for the Integrated Forecasting System](https://mlsanity.com/paper/2403.03702/) (2024)
- [PnP-DA: Towards Principled Plug-and-Play Integration of Variational Data Assimilation and Generative Models](https://mlsanity.com/paper/2508.00325/) (2025)
- [Sparsely-Supervised Data Assimilation via Physics-Informed Schr"odinger Bridge](https://mlsanity.com/paper/2603.22319/) (2026)
- [Accounting for model error in ensemble variational data assimilation through learned generative dynamics](https://mlsanity.com/paper/2610.17333/) (2026)

## Abstract (arXiv)

Recent studies have demonstrated that it is possible to combine machine learning with data assimilation to reconstruct the dynamics of a physical model partially and imperfectly observed. Data assimilation is used to estimate the system state from the observations, while machine learning computes a surrogate model of the dynamical system based on those estimated states. The surrogate model can be defined as an hybrid combination where a physical model based on prior knowledge is enhanced with a statistical model estimated by a neural network. The training of the neural network is typically done offline, once a large enough dataset of model state estimates is available. By contrast, with online approaches the surrogate model is improved each time a new system state estimate is computed. Online approaches naturally fit the sequential framework encountered in geosciences where new observations become available with time. In a recent methodology paper, we have developed a new weak-constraint 4D-Var formulation which can be used to train a neural network for online model error correction. In the present article, we develop a simplified version of that method, in the incremental 4D-Var framework adopted by most operational weather centres. The simplified method is implemented in the ECMWF Object-Oriented Prediction System, with the help of a newly developed Fortran neural network library, and tested with a two-layer two-dimensional quasi geostrophic model. The results confirm that online learning is effective and yields a more accurate model error correction than offline learning. Finally, the simplified method is compatible with future applications to state-of-the-art models such as the ECMWF Integrated Forecasting System.