# Efficient Online Variational Estimation via Monte Carlo Sampling

arXiv: https://arxiv.org/abs/2602.06579 · HTML: https://mlsanity.com/paper/2602.06579/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 31 references that are in the map.
- First posted on arXiv 6 Feb 2026.

## Builds on (most cited first, 18 of 18)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Monte Carlo Gradient Estimation in Machine Learning](https://mlsanity.com/paper/1906.10652/) (2019)
- [Structured Inference Networks for Nonlinear State Space Models](https://mlsanity.com/paper/1609.09869/) (2016)
- [Spectral Norm Regularization for Improving the Generalizability of Deep Learning](https://mlsanity.com/paper/1705.10941/) (2017)
- [Composing graphical models with neural networks for structured representations and fast inference](https://mlsanity.com/paper/1603.06277/) (2016)
- [Streaming Variational Bayes](https://mlsanity.com/paper/1307.6769/) (2013)
- [Non-asymptotic Analysis of Biased Stochastic Approximation Scheme](https://mlsanity.com/paper/1902.00629/) (2019)
- [Online Variational Filtering and Parameter Learning](https://mlsanity.com/paper/2110.13549/) (2021)
- [Variational Message Passing with Structured Inference Networks](https://mlsanity.com/paper/1803.05589/) (2018)
- [A General Method for Amortizing Variational Filtering](https://mlsanity.com/paper/1811.05090/) (2018)
- [Streaming Variational Monte Carlo](https://mlsanity.com/paper/1906.01549/) (2019)
- [A Review of Designs and Applications of Echo State Networks](https://mlsanity.com/paper/2012.02974/) (2020)
- [Online Variational Sequential Monte Carlo](https://mlsanity.com/paper/2312.12616/) (2023)
- [Real-Time Variational Method for Learning Neural Trajectory and its Dynamics](https://mlsanity.com/paper/2305.11278/) (2023)
- [Recursive Learning of Asymptotic Variational Objectives](https://mlsanity.com/paper/2411.02217/) (2024)
- [Parameter Estimation of State Space Models Using Particle Importance Sampling](https://mlsanity.com/paper/2502.00904/) (2025)

## Built on by (most cited first, 1 of 1)

- [PAC-Bayesian Reconstruction Guarantees for Time Series Variational Autoencoders](https://mlsanity.com/paper/2609.05212/) (2026)

## Abstract (arXiv)

This article addresses online variational estimation in parametric state-space models. We propose a new procedure for efficiently computing the evidence lower bound and its gradient in a streaming-data setting, where observations arrive sequentially. The algorithm allows for the simultaneous training of the model parameters and the distribution of the latent states given the observations. It is based on i.i.d. Monte Carlo sampling, coupled with a well-chosen deep architecture, enabling both computational efficiency and flexibility. The performance of the method is illustrated on both synthetic data and real-world air-quality data. The proposed approach is theoretically motivated by the existence of an asymptotic contrast function and the ergodicity of the underlying Markov chain, and applies more generally to the computation of additive expectations under posterior distributions in state-space models.