mlsanityData to 13 Oct 2026

Combining data assimilation and machine learning to emulate a dynamical model from sparse and noisy observations: a case study with the Lorenz 96 model

arXiv 2001.01520 ↗First posted 6 Jan 2020stat.ML cs.LG physics.ao-phcited by 72

Not on the map. Why?

Abstract (arXiv)

A novel method, based on the combination of data assimilation and machine learning is introduced. The new hybrid approach is designed for a two-fold scope: (i) emulating hidden, possibly chaotic, dynamics and (ii) predicting their future states. The method consists in applying iteratively a data assimilation step, here an ensemble Kalman filter, and a neural network. Data assimilation is used to optimally combine a surrogate model with sparse noisy data. The output analysis is spatially complete and is used as a training set by the neural network to update the surrogate model. The two steps are then repeated iteratively. Numerical experiments have been carried out using the chaotic 40-variables Lorenz 96 model, proving both convergence and statistical skill of the proposed hybrid approach. The surrogate model shows short-term forecast skill up to two Lyapunov times, the retrieval of positive Lyapunov exponents as well as the more energetic frequencies of the power density spectrum. The sensitivity of the method to critical setup parameters is also presented: the forecast skill decreases smoothly with increased observational noise but drops abruptly if less than half of the model domain is observed. The successful synergy between data assimilation and machine learning, proven here with a low-dimensional system, encourages further investigation of such hybrids with more sophisticated dynamics.

Key facts. Cited by 72 papers in the map (data through 13 Oct 2026). First posted on arXiv 6 Jan 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 17; 2022: 10; 2023: 13; 2024: 10; 2025: 11; 2026: 11.

72 papers.

Sort:

By area:

  1. 2023Score-based Data Assimilationcited by 95
  2. 2023Machine learning with data assimilation and uncertainty quantification for dynamical systems: a reviewcited by 74
  3. 2024ACE2: Accurately learning subseasonal to decadal atmospheric variability and forced responsescited by 54
  4. 2021A Framework for Machine Learning of Model Error in Dynamical Systemscited by 38
  5. 2021Latent Space Data Assimilation by using Deep Learningcited by 30
  6. 2021Variational Data Assimilation with a Learned Inverse Observation Operatorcited by 28
  7. 2021Auto-differentiable Ensemble Kalman Filterscited by 28
  8. 2022Generalised Latent Assimilation in Heterogeneous Reduced Spaces with Machine Learning Surrogate Modelscited by 26
  9. 2023FengWu-4DVar: Coupling the Data-driven Weather Forecasting Model with 4D Variational Assimilationcited by 22
  10. 2021Combining machine learning and data assimilation to forecast dynamical systems from noisy partial observationscited by 21
  11. 2021Integrating Recurrent Neural Networks with Data Assimilation for Scalable Data-Driven State Estimationcited by 21
  12. 2024FuXi Weather: A data-to-forecast machine learning system for global weathercited by 21
38 more
  1. 2024Fuxi-DA: A Generalized Deep Learning Data Assimilation Framework for Assimilating Satellite Observationscited by 18
  2. 2021Towards physically consistent data-driven weather forecasting: Integrating data assimilation with equivariance-preserving deep spatial transformerscited by 15
  3. 2021Bridging observation, theory and numerical simulation of the ocean using Machine Learningcited by 15
  4. 2023Reduced-Order Autodifferentiable Ensemble Kalman Filterscited by 14
  5. 2021A comparison of combined data assimilation and machine learning methods for offline and online model error correctioncited by 13
  6. 2022Online model error correction with neural networks in the incremental 4D-Var frameworkcited by 13
  7. 2022Flipped Classroom: Effective Teaching for Time Series Forecastingcited by 11
  8. 2021Learning to Assimilate in Chaotic Dynamical Systemscited by 10
  9. 2021Heterogeneous Stream-reservoir Graph Networks with Data Assimilationcited by 9
  10. 2022Deep learning delay coordinate dynamics for chaotic attractors from partial observable datacited by 8
  11. 2023FREE: The Foundational Semantic Recognition for Modeling Environmental Ecosystemscited by 8
  12. 2023Reconstruction, forecasting, and stability of chaotic dynamics from partial datacited by 7
  13. 2024Accurate deep learning-based filtering for chaotic dynamics by identifying instabilities without an ensemblecited by 7
  14. 2021State, global and local parameter estimation using local ensemble Kalman filters: applications to online machine learning of chaotic dynamicscited by 6
  15. 2021Discovery of interpretable structural model errors by combining Bayesian sparse regression and data assimilation: A chaotic Kuramoto-Sivashinsky test casecited by 6
  16. 2022Continuous-time identification of dynamic state-space models by deep subspace encodingcited by 6
  17. 2024CGNSDE: Conditional Gaussian Neural Stochastic Differential Equation for Modeling Complex Systems and Data Assimilationcited by 6
  18. 2024BI-EqNO: Generalized Approximate Bayesian Inference with an Equivariant Neural Operator Frameworkcited by 6
  19. 2022Bounded nonlinear forecasts of partially observed geophysical systems with physics-constrained deep learningcited by 5
  20. 2022Embed and Emulate: Learning to estimate parameters of dynamical systems with uncertainty quantificationcited by 5
  21. 2023Leveraging Neural Koopman Operators to Learn Continuous Representations of Dynamical Systems from Scarce Datacited by 5
  22. 2024TorchDA: A Python package for performing data assimilation with deep learning forward and transformation functionscited by 5
  23. 2025Ensemble Kalman filter in latent space using a variational autoencoder paircited by 5
  24. 2025Align-DA: Align Score-based Atmospheric Data Assimilation with Multiple Preferencescited by 5
  25. 2021Combining data assimilation and machine learning to estimate parameters of a convective-scale modelcited by 4
  26. 2021Joint calibration and mapping of satellite altimetry data using trainable variational modelscited by 3
  27. 2023Physics-Informed Long Short-Term Memory for Forecasting and Reconstruction of Chaoscited by 3
  28. 2023Learning Absorption Rates in Glucose-Insulin Dynamics from Meal Covariatescited by 3
  29. 2024Combined Optimization of Dynamics and Assimilation with End-to-End Learning on Sparse Observationscited by 3
  30. 2025Flow-based Bayesian filtering for high-dimensional nonlinear stochastic dynamical systemscited by 3
  31. 2026Latent Autoencoder Ensemble Kalman Filter for Nonlinear Data assimilationcited by 3
  32. 2021Machine learning-based conditional mean filter: a generalization of the ensemble Kalman filter for nonlinear data assimilationcited by 2
  33. 2021Adjoint-Matching Neural Network Surrogates for Fast 4D-Var Data Assimilationcited by 2
  34. 2023Online machine-learning forecast uncertainty estimation for sequential data assimilationcited by 2
  35. 2025RL-DAUNCE: Reinforcement Learning-Driven Data Assimilation with Uncertainty-Aware Constrained Ensemblescited by 2
  36. 2022A Transferable and Automatic Tuning of Deep Reinforcement Learning for Cost Effective Phishing Detectioncited by 1
  37. 2024Development of an offline and online hybrid model for the Integrated Forecasting Systemcited by 1
  38. 2024Long-time accuracy of ensemble Kalman filters for chaotic and machine-learned dynamical systemscited by 1

50 most cited of 72 All 72 (JSON)