mlsanityData to 14 Oct 2026

Bayesian inference of chaotic dynamics by merging data assimilation, machine learning and expectation-maximization

arXiv 2001.06270 ↗First posted 17 Jan 2020stat.ML cs.LG physics.ao-phcited by 38

Not on the map. Why?

Abstract (arXiv)

The reconstruction from observations of high-dimensional chaotic dynamics such as geophysical flows is hampered by (i) the partial and noisy observations that can realistically be obtained, (ii) the need to learn from long time series of data, and (iii) the unstable nature of the dynamics. To achieve such inference from the observations over long time series, it has been suggested to combine data assimilation and machine learning in several ways. We show how to unify these approaches from a Bayesian perspective using expectation-maximization and coordinate descents. In doing so, the model, the state trajectory and model error statistics are estimated all together. Implementations and approximations of these methods are discussed. Finally, we numerically and successfully test the approach on two relevant low-order chaotic models with distinct identifiability.

Key facts. Cited by 38 papers in the map (data through 14 Oct 2026). First posted on arXiv 17 Jan 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 8; 2022: 8; 2023: 5; 2024: 6; 2025: 6; 2026: 5.

38 papers.

Sort:

By area:

  1. 2023Machine learning with data assimilation and uncertainty quantification for dynamical systems: a reviewcited by 75
  2. 2021A Framework for Machine Learning of Model Error in Dynamical Systemscited by 38
  3. 2023Foundation Models for Weather and Climate Data Understanding: A Comprehensive Surveycited by 34
  4. 2021Auto-differentiable Ensemble Kalman Filterscited by 28
  5. 2021Combining machine learning and data assimilation to forecast dynamical systems from noisy partial observationscited by 21
  6. 2021Integrating Recurrent Neural Networks with Data Assimilation for Scalable Data-Driven State Estimationcited by 21
  7. 2022Online model error correction with neural networks in the incremental 4D-Var frameworkcited by 14
  8. 2023Reduced-Order Autodifferentiable Ensemble Kalman Filterscited by 14
  9. 2021A comparison of combined data assimilation and machine learning methods for offline and online model error correctioncited by 13
  10. 2023Fourier neural operator for real-time simulation of 3D dynamic urban microclimatecited by 12
  11. 2021Learning stochastic dynamical systems with neural networks mimicking the Euler-Maruyama schemecited by 10
  12. 2021State, global and local parameter estimation using local ensemble Kalman filters: applications to online machine learning of chaotic dynamicscited by 7
26 more
  1. 2024Accurate deep learning-based filtering for chaotic dynamics by identifying instabilities without an ensemblecited by 7
  2. 2024A Scalable Real-Time Data Assimilation Framework for Predicting Turbulent Atmosphere Dynamicscited by 5
  3. 2024TorchDA: A Python package for performing data assimilation with deep learning forward and transformation functionscited by 5
  4. 2025Ensemble Kalman filter in latent space using a variational autoencoder paircited by 5
  5. 2022Inversion of sea surface currents from satellite-derived SST-SSH synergies with 4DVarNetscited by 4
  6. 2022Learning Subgrid-scale Models with Neural Ordinary Differential Equationscited by 4
  7. 2022`Next Generation' Reservoir Computing: an Empirical Data-Driven Expression of Dynamical Equations in Time-Stepping Formcited by 3
  8. 2022Learning 4DVAR inversion directly from observationscited by 3
  9. 2024Combined Optimization of Dynamics and Assimilation with End-to-End Learning on Sparse Observationscited by 3
  10. 2021Machine learning-based conditional mean filter: a generalization of the ensemble Kalman filter for nonlinear data assimilationcited by 2
  11. 2022STEADY: Simultaneous State Estimation and Dynamics Learning from Indirect Observationscited by 2
  12. 2024Development of an offline and online hybrid model for the Integrated Forecasting Systemcited by 2
  13. 2025RL-DAUNCE: Reinforcement Learning-Driven Data Assimilation with Uncertainty-Aware Constrained Ensemblescited by 2
  14. 2022Ensemble forecasts in reproducing kernel Hilbert space familycited by 1
  15. 2024Long-time accuracy of ensemble Kalman filters for chaotic and machine-learned dynamical systemscited by 1
  16. 2025Reinforcement learning based data assimilation for unknown state modelcited by 1
  17. 2025Integrating Score-Based Diffusion Models with Machine Learning-Enhanced Localization for Advanced Data Assimilation in Geological Carbon Storagecited by 1
  18. 2025Balancing Accuracy and Speed: A Multi-Fidelity Ensemble Kalman Filter with a Machine Learning Surrogate Modelcited by 1
  19. 2026Pathwise Learning of Stochastic Dynamical Systems with Partial Observationscited by 1
  20. 2022Learning-based estimation of in-situ wind speed from underwater acousticscited by 0
  21. 2023Improving Surrogate Model Robustness to Perturbations for Dynamical Systems Through Machine Learning and Data Assimilationcited by 0
  22. 2025Hybrid twinning using PBDW and DeepONet for the effective state estimation and prediction on partially known systemscited by 0
  23. 2026Generative Model Proposal based Particle Filtering for Data Assimilationcited by 0
  24. 2026On a joint simultaneous learning of relevant feature subsets and subspaces in regression-like problemscited by 0
  25. 2026Variational Parameter Calibration with Physics-Aware Latent-Space Surrogatescited by 0
  26. 2026Accounting for model error in ensemble variational data assimilation through learned generative dynamicscited by 0