# Ensemble Kalman Filtering Meets Gaussian Process SSM for Non-Mean-Field and Online Inference

arXiv: https://arxiv.org/abs/2312.05910 · HTML: https://mlsanity.com/paper/2312.05910/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 60 references that are in the map.
- First posted on arXiv 10 Dec 2023.

## Builds on (most cited first, 30 of 33)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [Gaussian Processes for Big Data](https://mlsanity.com/paper/1309.6835/) (2013)
- [Stochastic Gradient Hamiltonian Monte Carlo](https://mlsanity.com/paper/1402.4102/) (2014)
- [Structured Inference Networks for Nonlinear State Space Models](https://mlsanity.com/paper/1609.09869/) (2016)
- [KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamics](https://mlsanity.com/paper/2107.10043/) (2021)
- [Gaussian Processes for Data-Efficient Learning in Robotics and Control](https://mlsanity.com/paper/1502.02860/) (2015)
- [Hilbert Space Methods for Reduced-Rank Gaussian Process Regression](https://mlsanity.com/paper/1401.5508/) (2014)
- [Probabilistic Recurrent State-Space Models](https://mlsanity.com/paper/1801.10395/) (2018)
- [Variational Gaussian Process State-Space Models](https://mlsanity.com/paper/1406.4905/) (2014)
- [Identification of Gaussian Process State Space Models](https://mlsanity.com/paper/1705.10888/) (2017)
- [Auto-differentiable Ensemble Kalman Filters](https://mlsanity.com/paper/2107.07687/) (2021)
- [Bayesian Inference and Learning in Gaussian Process State-Space Models with Particle MCMC](https://mlsanity.com/paper/1306.2861/) (2013)
- [Overcoming Mean-Field Approximations in Recurrent Gaussian Process Models](https://mlsanity.com/paper/1906.05828/) (2019)
- [Online Variational Filtering and Parameter Learning](https://mlsanity.com/paper/2110.13549/) (2021)
- [Computationally Efficient Bayesian Learning of Gaussian Process State Space Models](https://mlsanity.com/paper/1506.02267/) (2015)
- [A flexible state space model for learning nonlinear dynamical systems](https://mlsanity.com/paper/1603.05486/) (2016)
- [The Ensemble Kalman Filter: A Signal Processing Perspective](https://mlsanity.com/paper/1702.08061/) (2017)
- [Reduced-Order Autodifferentiable Ensemble Kalman Filters](https://mlsanity.com/paper/2301.11961/) (2023)
- [Rethinking Bayesian Learning for Data Analysis: The Art of Prior and Inference in Sparsity-Aware Modeling](https://mlsanity.com/paper/2205.14283/) (2022)
- [Streaming Variational Monte Carlo](https://mlsanity.com/paper/1906.01549/) (2019)
- [A Joint introduction to Gaussian Processes and Relevance Vector Machines with Connections to Kalman filtering and other Kernel Smoothers](https://mlsanity.com/paper/2009.09217/) (2020)
- [Linear Multiple Low-Rank Kernel Based Stationary Gaussian Processes Regression for Time Series](https://mlsanity.com/paper/1904.09559/) (2019)
- [Online Joint State Inference and Learning of Partially Unknown State-Space Models](https://mlsanity.com/paper/2102.07418/) (2021)
- [Efficient Learning of the Parameters of Non-Linear Models using Differentiable Resampling in Particle Filters](https://mlsanity.com/paper/2111.01409/) (2021)
- [High-dimensional Filtering using Nested Sequential Monte Carlo](https://mlsanity.com/paper/1612.09162/) (2016)
- [Free-Form Variational Inference for Gaussian Process State-Space Models](https://mlsanity.com/paper/2302.09921/) (2023)
- [Output-Dependent Gaussian Process State-Space Model](https://mlsanity.com/paper/2212.07608/) (2022)
- [Towards Flexibility and Interpretability of Gaussian Process State-Space Model](https://mlsanity.com/paper/2301.08843/) (2023)

## Built on by (most cited first, 5 of 5)

- [Regularization-Based Efficient Continual Learning in Deep State-Space Models](https://mlsanity.com/paper/2403.10123/) (2024)
- [Efficient Transformed Gaussian Process State-Space Models for Non-Stationary High-Dimensional Dynamical Systems](https://mlsanity.com/paper/2503.18309/) (2025)
- [Recursive Gaussian Process State Space Model](https://mlsanity.com/paper/2411.14679/) (2024)
- [Robust Bayesian Optimization with Q-Exponential Surrogates](https://mlsanity.com/paper/2609.32775/) (2026)
- [Simulation-Free Learning of GP-SDEs from Irregular Observations](https://mlsanity.com/paper/2609.33112/) (2026)

## Abstract (arXiv)

The Gaussian process state-space models (GPSSMs) represent a versatile class of data-driven nonlinear dynamical system models. However, the presence of numerous latent variables in GPSSM incurs unresolved issues for existing variational inference approaches, particularly under the more realistic non-mean-field (NMF) assumption, including extensive training effort, compromised inference accuracy, and infeasibility for online applications, among others. In this paper, we tackle these challenges by incorporating the ensemble Kalman filter (EnKF), a well-established model-based filtering technique, into the NMF variational inference framework to approximate the posterior distribution of the latent states. This novel marriage between EnKF and GPSSM not only eliminates the need for extensive parameterization in learning variational distributions, but also enables an interpretable, closed-form approximation of the evidence lower bound (ELBO). Moreover, owing to the streamlined parameterization via the EnKF, the new GPSSM model can be easily accommodated in online learning applications. We demonstrate that the resulting EnKF-aided online algorithm embodies a principled objective function by ensuring data-fitting accuracy while incorporating model regularizations to mitigate overfitting. We also provide detailed analysis and fresh insights for the proposed algorithms. Comprehensive evaluation across diverse real and synthetic datasets corroborates the superior learning and inference performance of our EnKF-aided variational inference algorithms compared to existing methods.