mlsanityData to 7 Oct 2026

Ensemble Kalman Filtering Meets Gaussian Process SSM for Non-Mean-Field and Online Inference

arXiv 2312.05910 ↗First posted 10 Dec 2023cs.LG eess.SP stat.MLcited by 5

Not on the map. Why?

Abstract (arXiv)

The Gaussian process state-space models (GPSSMs) represent a versatile class of data-driven nonlinear dynamical system models. However, the presence of numerous latent variables in GPSSM incurs unresolved issues for existing variational inference approaches, particularly under the more realistic non-mean-field (NMF) assumption, including extensive training effort, compromised inference accuracy, and infeasibility for online applications, among others. In this paper, we tackle these challenges by incorporating the ensemble Kalman filter (EnKF), a well-established model-based filtering technique, into the NMF variational inference framework to approximate the posterior distribution of the latent states. This novel marriage between EnKF and GPSSM not only eliminates the need for extensive parameterization in learning variational distributions, but also enables an interpretable, closed-form approximation of the evidence lower bound (ELBO). Moreover, owing to the streamlined parameterization via the EnKF, the new GPSSM model can be easily accommodated in online learning applications. We demonstrate that the resulting EnKF-aided online algorithm embodies a principled objective function by ensuring data-fitting accuracy while incorporating model regularizations to mitigate overfitting. We also provide detailed analysis and fresh insights for the proposed algorithms. Comprehensive evaluation across diverse real and synthetic datasets corroborates the superior learning and inference performance of our EnKF-aided variational inference algorithms compared to existing methods.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 33 of its 60 references that are in the map. First posted on arXiv 10 Dec 2023.

Builds on

33 of 60 references.

References · 33

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2017A Brief Survey of Deep Reinforcement Learningcited by 466
  4. 2013Gaussian Processes for Big Datacited by 377
  5. 2014Stochastic Gradient Hamiltonian Monte Carlocited by 341
  6. 2016Structured Inference Networks for Nonlinear State Space Modelscited by 182
  7. 2021KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamicscited by 128
  8. 2015Gaussian Processes for Data-Efficient Learning in Robotics and Controlcited by 97
  9. 2014Hilbert Space Methods for Reduced-Rank Gaussian Process Regressioncited by 88
  10. 2018Probabilistic Recurrent State-Space Modelscited by 64
  11. 2014Variational Gaussian Process State-Space Modelscited by 43
  12. 2017Identification of Gaussian Process State Space Modelscited by 31
21 more
  1. 2021Auto-differentiable Ensemble Kalman Filterscited by 28
  2. 2013Bayesian Inference and Learning in Gaussian Process State-Space Models with Particle MCMCcited by 24
  3. 2019Overcoming Mean-Field Approximations in Recurrent Gaussian Process Modelscited by 20
  4. 2021Online Variational Filtering and Parameter Learningcited by 19
  5. 2015Computationally Efficient Bayesian Learning of Gaussian Process State Space Modelscited by 18
  6. 2016A flexible state space model for learning nonlinear dynamical systemscited by 18
  7. 2017The Ensemble Kalman Filter: A Signal Processing Perspectivecited by 17
  8. 2022Rethinking Bayesian Learning for Data Analysis: The Art of Prior and Inference in Sparsity-Aware Modelingcited by 12
  9. 2023Reduced-Order Autodifferentiable Ensemble Kalman Filterscited by 12
  10. 2019Streaming Variational Monte Carlocited by 11
  11. 2020A Joint introduction to Gaussian Processes and Relevance Vector Machines with Connections to Kalman filtering and other Kernel Smootherscited by 10
  12. 2019Linear Multiple Low-Rank Kernel Based Stationary Gaussian Processes Regression for Time Seriescited by 9
  13. 2021Online Joint State Inference and Learning of Partially Unknown State-Space Modelscited by 9
  14. 2021Efficient Learning of the Parameters of Non-Linear Models using Differentiable Resampling in Particle Filterscited by 9
  15. 2016High-dimensional Filtering using Nested Sequential Monte Carlocited by 8
  16. 2023Free-Form Variational Inference for Gaussian Process State-Space Modelscited by 7
  17. 2022Output-Dependent Gaussian Process State-Space Modelcited by 6
  18. 2023Sequential Estimation of Gaussian Process-based Deep State-Space Modelscited by 5
  19. 2020Variational System Identification for Nonlinear State-Space Modelscited by 4
  20. 2023Towards Flexibility and Interpretability of Gaussian Process State-Space Modelcited by 4
  21. 2020Gaussian Variational State Estimation for Nonlinear State-Space Modelscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 2.

5 papers.

  1. 2024Regularization-Based Efficient Continual Learning in Deep State-Space Modelscited by 4
  2. 2025Efficient Transformed Gaussian Process State-Space Models for Non-Stationary High-Dimensional Dynamical Systemscited by 2
  3. 2024Recursive Gaussian Process State Space Modelcited by 0
  4. 2026Robust Bayesian Optimization with Q-Exponential Surrogatescited by 0
  5. 2026Simulation-Free Learning of GP-SDEs from Irregular Observationscited by 0