# Reduced-Order Autodifferentiable Ensemble Kalman Filters

arXiv: https://arxiv.org/abs/2301.11961 · HTML: https://mlsanity.com/paper/2301.11961/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 101 references that are in the map.
- First posted on arXiv 27 Jan 2023.

## Builds on (most cited first, 30 of 35)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Neural Controlled Differential Equations for Irregular Time Series](https://mlsanity.com/paper/2005.08926/) (2020)
- [Deep learning for universal linear embeddings of nonlinear dynamics](https://mlsanity.com/paper/1712.09707/) (2017)
- [GRU-ODE-Bayes: Continuous modeling of sporadically-observed time series](https://mlsanity.com/paper/1905.12374/) (2019)
- [A Critical Review of Recurrent Neural Networks for Sequence Learning](https://mlsanity.com/paper/1506.00019/) (2015)
- [Variational Sequential Monte Carlo](https://mlsanity.com/paper/1705.11140/) (2017)
- [Filtering Variational Objectives](https://mlsanity.com/paper/1705.09279/) (2017)
- [Learning Long-Term Dependencies in Irregularly-Sampled Time Series](https://mlsanity.com/paper/2006.04418/) (2020)
- [Combining data assimilation and machine learning to emulate a dynamical model from sparse and noisy observations: a case study with the Lorenz 96 model](https://mlsanity.com/paper/2001.01520/) (2020)
- [Auto-Encoding Sequential Monte Carlo](https://mlsanity.com/paper/1705.10306/) (2017)
- [Differentiable Particle Filtering via Entropy-Regularized Optimal Transport](https://mlsanity.com/paper/2102.07850/) (2021)
- [Deep convolutional recurrent autoencoders for learning low-dimensional feature dynamics of fluid systems](https://mlsanity.com/paper/1808.01346/) (2018)
- [Extracting Sparse High-Dimensional Dynamics from Limited Data](https://mlsanity.com/paper/1707.08528/) (2017)
- [Inference via low-dimensional couplings](https://mlsanity.com/paper/1703.06131/) (2017)
- [Using machine learning to correct model error in data assimilation and forecast applications](https://mlsanity.com/paper/2010.12605/) (2020)
- [Bayesian inference of chaotic dynamics by merging data assimilation, machine learning and expectation-maximization](https://mlsanity.com/paper/2001.06270/) (2020)
- [A Framework for Machine Learning of Model Error in Dynamical Systems](https://mlsanity.com/paper/2107.06658/) (2021)
- [Machine Learning for Prediction with Missing Dynamics](https://mlsanity.com/paper/1910.05861/) (2019)
- [Auto-differentiable Ensemble Kalman Filters](https://mlsanity.com/paper/2107.07687/) (2021)
- [Combining Machine Learning with Knowledge-Based Modeling for Scalable Forecasting and Subgrid-Scale Closure of Large, Complex, Spatiotemporal Systems](https://mlsanity.com/paper/2002.05514/) (2020)
- [Inverse Problems and Data Assimilation](https://mlsanity.com/paper/1810.06191/) (2018)
- [Gated recurrent units viewed through the lens of continuous time dynamical systems](https://mlsanity.com/paper/1906.01005/) (2019)
- [The Ensemble Kalman Filter: A Signal Processing Perspective](https://mlsanity.com/paper/1702.08061/) (2017)
- [Tikhonov Regularization Within Ensemble Kalman Inversion](https://mlsanity.com/paper/1901.10382/) (2019)
- [Iterative Ensemble Kalman Methods: A Unified Perspective with Some New Variants](https://mlsanity.com/paper/2010.13299/) (2020)
- [EM-like Learning Chaotic Dynamics from Noisy and Partial Observations](https://mlsanity.com/paper/1903.10335/) (2019)

## Built on by (most cited first, 14 of 14)

- [Latent-EnSF: A Latent Ensemble Score Filter for High-Dimensional Data Assimilation with Sparse Observation Data](https://mlsanity.com/paper/2409.00127/) (2024)
- [Sparse identification of nonlinear dynamics and Koopman operators with Shallow Recurrent Decoder Networks](https://mlsanity.com/paper/2501.13329/) (2025)
- [LD-EnSF: Synergizing Latent Dynamics with Ensemble Score Filters for Fast Data Assimilation with Sparse Observations](https://mlsanity.com/paper/2411.19305/) (2024)
- [Efficient Bayesian Physics Informed Neural Networks for Inverse Problems via Ensemble Kalman Inversion](https://mlsanity.com/paper/2303.07392/) (2023)
- [Deep Bayesian Filter for Bayes-faithful Data Assimilation](https://mlsanity.com/paper/2405.18674/) (2024)
- [Modeling Partially Observed Nonlinear Dynamical Systems and Efficient Data Assimilation via Discrete-Time Conditional Gaussian Koopman Network](https://mlsanity.com/paper/2507.08749/) (2025)
- [Ensemble Kalman Filtering Meets Gaussian Process SSM for Non-Mean-Field and Online Inference](https://mlsanity.com/paper/2312.05910/) (2023)
- [The Deep Latent Space Particle Filter for Real-Time Data Assimilation with Uncertainty Quantification](https://mlsanity.com/paper/2406.02204/) (2024)
- [Flow-based Bayesian filtering for high-dimensional nonlinear stochastic dynamical systems](https://mlsanity.com/paper/2502.16232/) (2025)
- [iPINNER: An Iterative Physics-Informed Neural Network with Ensemble Kalman Filter](https://mlsanity.com/paper/2506.00731/) (2025)
- [Auto-differentiable data assimilation: Co-learning of states, dynamics, and filtering algorithms](https://mlsanity.com/paper/2603.20891/) (2026)
- [Feature-preserving Latent-EnKF for Data Assimilation of Flows with Shocks](https://mlsanity.com/paper/2606.12559/) (2026)
- [A Unified Framework for Bayesian Data Assimilation with Generative Models and Observation Interpolants](https://mlsanity.com/paper/2610.03396/) (2026)
- [LD-EnFF: Latent-Dynamics Ensemble Flow Filtering for Data Assimilation with Sparse Observations](https://mlsanity.com/paper/2610.04034/) (2026)

## Abstract (arXiv)

This paper introduces a computational framework to reconstruct and forecast a partially observed state that evolves according to an unknown or expensive-to-simulate dynamical system. Our reduced-order autodifferentiable ensemble Kalman filters (ROAD-EnKFs) learn a latent low-dimensional surrogate model for the dynamics and a decoder that maps from the latent space to the state space. The learned dynamics and decoder are then used within an ensemble Kalman filter to reconstruct and forecast the state. Numerical experiments show that if the state dynamics exhibit a hidden low-dimensional structure, ROAD-EnKFs achieve higher accuracy at lower computational cost compared to existing methods. If such structure is not expressed in the latent state dynamics, ROAD-EnKFs achieve similar accuracy at lower cost, making them a promising approach for surrogate state reconstruction and forecasting.