mlsanityData to 7 Oct 2026

Integrating Recurrent Neural Networks with Data Assimilation for Scalable Data-Driven State Estimation

arXiv 2109.12269 ↗First posted 25 Sep 2021cs.LG cs.AI math.DScited by 20

Not on the map. Why?

Abstract (arXiv)

Data assimilation (DA) is integrated with machine learning in order to perform entirely data-driven online state estimation. To achieve this, recurrent neural networks (RNNs) are implemented as surrogate models to replace key components of the DA cycle in numerical weather prediction (NWP), including the conventional numerical forecast model, the forecast error covariance matrix, and the tangent linear and adjoint models. It is shown how these RNNs can be initialized using DA methods to directly update the hidden/reservoir state with observations of the target system. The results indicate that these techniques can be applied to estimate the state of a system for the repeated initialization of short-term forecasts, even in the absence of a traditional numerical forecast model. Further, it is demonstrated how these integrated RNN-DA methods can scale to higher dimensions by applying domain localization and parallelization, providing a path for practical applications in NWP.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 10 of its 69 references that are in the map. First posted on arXiv 25 Sep 2021.

Builds on

10 of 69 references.

References · 10

  1. 2019Backpropagation Algorithms and Reservoir Computing in Recurrent Neural Networks for the Forecasting of Complex Spatiotemporal Dynamicscited by 132
  2. 2020Combining data assimilation and machine learning to emulate a dynamical model from sparse and noisy observations: a case study with the Lorenz 96 modelcited by 72
  3. 2019Forecasting Chaotic Systems with Very Low Connectivity Reservoir Computerscited by 43
  4. 2020Bayesian inference of chaotic dynamics by merging data assimilation, machine learning and expectation-maximizationcited by 37
  5. 2021Latent Space Data Assimilation by using Deep Learningcited by 29
  6. 2020Online learning of both state and dynamics using ensemble Kalman filterscited by 15
  7. 2017Machine Learning, Deepest Learning: Statistical Data Assimilation Problemscited by 13
  8. 2021A comparison of combined data assimilation and machine learning methods for offline and online model error correctioncited by 13
  9. 2016Degenerate Kalman filter error covariances and their convergence onto the unstable subspacecited by 3
  10. 2021Forecasting Using Reservoir Computing: The Role of Generalized Synchronizationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 4; 2024: 9; 2025: 3; 2026: 2.

20 papers.

Sort:

By area:

  1. 2023Towards an end-to-end artificial intelligence driven global weather forecasting systemcited by 20
  2. 2022Deep learning-enhanced ensemble-based data assimilation for high-dimensional nonlinear dynamical systemscited by 12
  3. 2024LD-EnSF: Synergizing Latent Dynamics with Ensemble Score Filters for Fast Data Assimilation with Sparse Observationscited by 10
  4. 2023Latent assimilation with implicit neural representations for unknown dynamicscited by 8
  5. 2024FNP: Fourier Neural Processes for Arbitrary-Resolution Data Assimilationcited by 8
  6. 2024State-observation augmented diffusion model for nonlinear assimilation with unknown dynamicscited by 7
  7. 2025Learning dynamical systems with hit-and-run random feature mapscited by 7
  8. 2023Constraining Chaos: Enforcing dynamical invariants in the training of recurrent neural networkscited by 6
  9. 2024TorchDA: A Python package for performing data assimilation with deep learning forward and transformation functionscited by 5
  10. 20244D-Var using Hessian approximation and backpropagation applied to automatically-differentiable numerical and machine learning modelscited by 4
  11. 2022`Next Generation' Reservoir Computing: an Empirical Data-Driven Expression of Dynamical Equations in Time-Stepping Formcited by 3
  12. 2023Temporal Subsampling Diminishes Small Spatial Scales in Recurrent Neural Network Emulators of Geophysical Turbulencecited by 3
8 more
  1. 2024Combined Optimization of Dynamics and Assimilation with End-to-End Learning on Sparse Observationscited by 2
  2. 2025RL-DAUNCE: Reinforcement Learning-Driven Data Assimilation with Uncertainty-Aware Constrained Ensemblescited by 2
  3. 2024Ensemble data assimilation to diagnose AI-based weather prediction model: A case with ClimaX version 0.3.1cited by 1
  4. 2024Long-time accuracy of ensemble Kalman filters for chaotic and machine-learned dynamical systemscited by 1
  5. 2026ADDA: a Modular Framework for Representing, Simulating and Assimilating Dynamics with End-to-end Differentiabilitycited by 1
  6. 2024U-Net Kalman Filter (UNetKF): An Example of Machine Learning-assisted Ensemble Data Assimilationcited by 0
  7. 2025Probabilistic Emulation of the Community Radiative Transfer Model Using Machine Learningcited by 0
  8. 2026Ensemble Controlled-Flow Filtering for Implicit Data Assimilationcited by 0