# Online Variational Filtering and Parameter Learning

arXiv: https://arxiv.org/abs/2110.13549 · HTML: https://mlsanity.com/paper/2110.13549/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 45 references that are in the map.
- First posted on arXiv 26 Oct 2021.

## Builds on (most cited first, 23 of 23)

- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review](https://mlsanity.com/paper/1805.00909/) (2018)
- [A Recurrent Latent Variable Model for Sequential Data](https://mlsanity.com/paper/1506.02216/) (2015)
- [Structured Inference Networks for Nonlinear State Space Models](https://mlsanity.com/paper/1609.09869/) (2016)
- [Expectation Propagation for approximate Bayesian inference](https://mlsanity.com/paper/1301.2294/) (2013)
- [Disentangled Sequential Autoencoder](https://mlsanity.com/paper/1803.02991/) (2018)
- [Sequential Neural Models with Stochastic Layers](https://mlsanity.com/paper/1605.07571/) (2016)
- [Variational Sequential Monte Carlo](https://mlsanity.com/paper/1705.11140/) (2017)
- [Filtering Variational Objectives](https://mlsanity.com/paper/1705.09279/) (2017)
- [Auto-Encoding Sequential Monte Carlo](https://mlsanity.com/paper/1705.10306/) (2017)
- [Differentiable Particle Filters: End-to-End Learning with Algorithmic Priors](https://mlsanity.com/paper/1805.11122/) (2018)
- [Differentiable Particle Filtering via Entropy-Regularized Optimal Transport](https://mlsanity.com/paper/2102.07850/) (2021)
- [Temporal Difference Variational Auto-Encoder](https://mlsanity.com/paper/1806.03107/) (2018)
- [On Particle Methods for Parameter Estimation in State-Space Models](https://mlsanity.com/paper/1412.8695/) (2014)
- [A Unified Framework of Online Learning Algorithms for Training Recurrent Neural Networks](https://mlsanity.com/paper/1907.02649/) (2019)
- [Black box variational inference for state space models](https://mlsanity.com/paper/1511.07367/) (2015)
- [Discriminative Particle Filter Reinforcement Learning for Complex Partial Observations](https://mlsanity.com/paper/2002.09884/) (2020)
- [VIREL: A Variational Inference Framework for Reinforcement Learning](https://mlsanity.com/paper/1811.01132/) (2018)
- [A General Method for Amortizing Variational Filtering](https://mlsanity.com/paper/1811.05090/) (2018)
- [Streaming Variational Monte Carlo](https://mlsanity.com/paper/1906.01549/) (2019)
- [The Neural Moving Average Model for Scalable Variational Inference of State Space Models](https://mlsanity.com/paper/1910.00879/) (2019)
- [Amortised Learning by Wake-Sleep](https://mlsanity.com/paper/2002.09737/) (2020)
- [Variational State and Parameter Estimation](https://mlsanity.com/paper/2012.07269/) (2020)

## Built on by (most cited first, 19 of 19)

- [Learning Optimal Filters Using Variational Inference](https://mlsanity.com/paper/2406.18066/) (2024)
- [Knowledge Adaptation as Posterior Correction](https://mlsanity.com/paper/2506.14262/) (2025)
- [Ensemble Kalman Filtering Meets Gaussian Process SSM for Non-Mean-Field and Online Inference](https://mlsanity.com/paper/2312.05910/) (2023)
- [Online Variational Sequential Monte Carlo](https://mlsanity.com/paper/2312.12616/) (2023)
- [Real-Time Variational Method for Learning Neural Trajectory and its Dynamics](https://mlsanity.com/paper/2305.11278/) (2023)
- [Variational excess risk bound for general state space models](https://mlsanity.com/paper/2312.09607/) (2023)
- [Regime Learning for Differentiable Particle Filters](https://mlsanity.com/paper/2405.04865/) (2024)
- [Tree-based variational inference for Poisson log-normal models](https://mlsanity.com/paper/2406.17361/) (2024)
- [A Metalearned Neural Circuit for Nonparametric Bayesian Inference](https://mlsanity.com/paper/2311.14601/) (2023)
- [Importance sampling for online variational learning](https://mlsanity.com/paper/2402.02859/) (2024)
- [Efficient Online Variational Estimation via Monte Carlo Sampling](https://mlsanity.com/paper/2602.06579/) (2026)
- [PAC-Bayesian Reconstruction Guarantees for Time Series Variational Autoencoders](https://mlsanity.com/paper/2609.05212/) (2026)
- [Linear Variational State-Space Filtering](https://mlsanity.com/paper/2201.01353/) (2022)
- [Amortized backward variational inference in nonlinear state-space models](https://mlsanity.com/paper/2206.00319/) (2022)
- [Brain in the Dark: Design Principles for Neuro-mimetic Learning and Inference](https://mlsanity.com/paper/2307.08613/) (2023)
- [ODEStream: A Buffer-Free Online Learning Framework with ODE-based Adaptor for Streaming Time Series Forecasting](https://mlsanity.com/paper/2411.07413/) (2024)
- [Recursive Entropic Variational Inference for Nonlinear State-Space Models](https://mlsanity.com/paper/2511.15409/) (2025)
- [Online Generalised Predictive Coding](https://mlsanity.com/paper/2605.02675/) (2026)
- [PR-Smoother: Simulator-Preserving Non-Gaussian Smoothing for Data Assimilation](https://mlsanity.com/paper/2609.26890/) (2026)

## Abstract (arXiv)

We present a variational method for online state estimation and parameter learning in state-space models (SSMs), a ubiquitous class of latent variable models for sequential data. As per standard batch variational techniques, we use stochastic gradients to simultaneously optimize a lower bound on the log evidence with respect to both model parameters and a variational approximation of the states' posterior distribution. However, unlike existing approaches, our method is able to operate in an entirely online manner, such that historic observations do not require revisitation after being incorporated and the cost of updates at each time step remains constant, despite the growing dimensionality of the joint posterior distribution of the states. This is achieved by utilizing backward decompositions of this joint posterior distribution and of its variational approximation, combined with Bellman-type recursions for the evidence lower bound and its gradients. We demonstrate the performance of this methodology across several examples, including high-dimensional SSMs and sequential Variational Auto-Encoders.