mlsanityData to 7 Oct 2026

Reduced-Order Autodifferentiable Ensemble Kalman Filters

arXiv 2301.11961 ↗First posted 27 Jan 2023stat.ML cs.LG math.DScited by 12

Not on the map. Why?

Abstract (arXiv)

This paper introduces a computational framework to reconstruct and forecast a partially observed state that evolves according to an unknown or expensive-to-simulate dynamical system. Our reduced-order autodifferentiable ensemble Kalman filters (ROAD-EnKFs) learn a latent low-dimensional surrogate model for the dynamics and a decoder that maps from the latent space to the state space. The learned dynamics and decoder are then used within an ensemble Kalman filter to reconstruct and forecast the state. Numerical experiments show that if the state dynamics exhibit a hidden low-dimensional structure, ROAD-EnKFs achieve higher accuracy at lower computational cost compared to existing methods. If such structure is not expressed in the latent state dynamics, ROAD-EnKFs achieve similar accuracy at lower cost, making them a promising approach for surrogate state reconstruction and forecasting.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 35 of its 101 references that are in the map. First posted on arXiv 27 Jan 2023.

Builds on

35 of 101 references.

References · 35

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2018Neural Ordinary Differential Equationscited by 3,733
  4. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  5. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  6. 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 501
  7. 2017Deep learning for universal linear embeddings of nonlinear dynamicscited by 491
  8. 2019GRU-ODE-Bayes: Continuous modeling of sporadically-observed time seriescited by 230
  9. 2015A Critical Review of Recurrent Neural Networks for Sequence Learningcited by 223
  10. 2017Variational Sequential Monte Carlocited by 93
  11. 2017Filtering Variational Objectivescited by 91
  12. 2020Learning Long-Term Dependencies in Irregularly-Sampled Time Seriescited by 82
23 more
  1. 2020Combining data assimilation and machine learning to emulate a dynamical model from sparse and noisy observations: a case study with the Lorenz 96 modelcited by 72
  2. 2017Auto-Encoding Sequential Monte Carlocited by 64
  3. 2021Differentiable Particle Filtering via Entropy-Regularized Optimal Transportcited by 59
  4. 2018Deep convolutional recurrent autoencoders for learning low-dimensional feature dynamics of fluid systemscited by 54
  5. 2017Extracting Sparse High-Dimensional Dynamics from Limited Datacited by 50
  6. 2017Inference via low-dimensional couplingscited by 43
  7. 2020Using machine learning to correct model error in data assimilation and forecast applicationscited by 40
  8. 2020Bayesian inference of chaotic dynamics by merging data assimilation, machine learning and expectation-maximizationcited by 37
  9. 2021A Framework for Machine Learning of Model Error in Dynamical Systemscited by 37
  10. 2021Auto-differentiable Ensemble Kalman Filterscited by 28
  11. 2019Machine Learning for Prediction with Missing Dynamicscited by 27
  12. 2020Combining Machine Learning with Knowledge-Based Modeling for Scalable Forecasting and Subgrid-Scale Closure of Large, Complex, Spatiotemporal Systemscited by 23
  13. 2018Inverse Problems and Data Assimilationcited by 18
  14. 2019Gated recurrent units viewed through the lens of continuous time dynamical systemscited by 18
  15. 2017The Ensemble Kalman Filter: A Signal Processing Perspectivecited by 17
  16. 2019Tikhonov Regularization Within Ensemble Kalman Inversioncited by 13
  17. 2020Iterative Ensemble Kalman Methods: A Unified Perspective with Some New Variantscited by 13
  18. 2019EM-like Learning Chaotic Dynamics from Noisy and Partial Observationscited by 11
  19. 2021Machine Learning Techniques to Construct Patched Analog Ensembles for Data Assimilationcited by 7
  20. 2022Non-Asymptotic Analysis of Ensemble Kalman Updates: Effective Dimension and Localizationcited by 7
  21. 2019Ensemble MCMC: Accelerating Pseudo-Marginal MCMC for State Space Models using the Ensemble Kalman Filtercited by 4
  22. 2016A Bayesian adaptive ensemble Kalman filter for sequential state and parameter estimationcited by 3
  23. 2017Estimating model evidence using ensemble-based data assimilation with localization - The model selection problemcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 4; 2025: 4; 2026: 2.

12 papers.

  1. 2024Latent-EnSF: A Latent Ensemble Score Filter for High-Dimensional Data Assimilation with Sparse Observation Datacited by 20
  2. 2025Sparse identification of nonlinear dynamics and Koopman operators with Shallow Recurrent Decoder Networkscited by 19
  3. 2024LD-EnSF: Synergizing Latent Dynamics with Ensemble Score Filters for Fast Data Assimilation with Sparse Observationscited by 10
  4. 2023Efficient Bayesian Physics Informed Neural Networks for Inverse Problems via Ensemble Kalman Inversioncited by 6
  5. 2025Modeling Partially Observed Nonlinear Dynamical Systems and Efficient Data Assimilation via Discrete-Time Conditional Gaussian Koopman Networkcited by 6
  6. 2023Ensemble Kalman Filtering Meets Gaussian Process SSM for Non-Mean-Field and Online Inferencecited by 5
  7. 2024Deep Bayesian Filter for Bayes-faithful Data Assimilationcited by 5
  8. 2025Flow-based Bayesian filtering for high-dimensional nonlinear stochastic dynamical systemscited by 3
  9. 2025iPINNER: An Iterative Physics-Informed Neural Network with Ensemble Kalman Filtercited by 3
  10. 2024The Deep Latent Space Particle Filter for Real-Time Data Assimilation with Uncertainty Quantificationcited by 1
  11. 2026Auto-differentiable data assimilation: Co-learning of states, dynamics, and filtering algorithmscited by 1
  12. 2026Feature-preserving Latent-EnKF for Data Assimilation of Flows with Shockscited by 0