# Black box variational inference for state space models

arXiv: https://arxiv.org/abs/1511.07367 · HTML: https://mlsanity.com/paper/1511.07367/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Nov 2015.

## Built on by (most cited first, 30 of 31)

- [KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamics](https://mlsanity.com/paper/2107.10043/) (2021)
- [Score-based Data Assimilation](https://mlsanity.com/paper/2306.10574/) (2023)
- [Differentiable Particle Filtering via Entropy-Regularized Optimal Transport](https://mlsanity.com/paper/2102.07850/) (2021)
- [Tractable Dendritic RNNs for Reconstructing Nonlinear Dynamical Systems](https://mlsanity.com/paper/2207.02542/) (2022)
- [Batch and match: black-box variational inference with a score-based divergence](https://mlsanity.com/paper/2402.14758/) (2024)
- [Online Variational Filtering and Parameter Learning](https://mlsanity.com/paper/2110.13549/) (2021)
- [Unsupervised Learned Kalman Filtering](https://mlsanity.com/paper/2110.09005/) (2021)
- [Reconstructing Nonlinear Dynamical Systems from Multi-Modal Time Series](https://mlsanity.com/paper/2111.02922/) (2021)
- [Mind the Gap when Conditioning Amortised Inference in Sequential Latent-Variable Models](https://mlsanity.com/paper/2101.07046/) (2021)
- [On Uncertainty in Deep State Space Models for Model-Based Reinforcement Learning](https://mlsanity.com/paper/2210.09256/) (2022)
- [Physics-aware, probabilistic model order reduction with guaranteed stability](https://mlsanity.com/paper/2101.05834/) (2021)
- [Revisiting Structured Variational Autoencoders](https://mlsanity.com/paper/2305.16543/) (2023)
- [Modeling Latent Neural Dynamics with Gaussian Process Switching Linear Dynamical Systems](https://mlsanity.com/paper/2408.03330/) (2024)
- [KalMamba: Towards Efficient Probabilistic State Space Models for RL under Uncertainty](https://mlsanity.com/paper/2406.15131/) (2024)
- [GD-VAEs: Geometric Dynamic Variational Autoencoders for Learning Nonlinear Dynamics and Dimension Reductions](https://mlsanity.com/paper/2206.05183/) (2022)
- [Knowledge Adaptation as Posterior Correction](https://mlsanity.com/paper/2506.14262/) (2025)
- [Provably Scalable Black-Box Variational Inference with Structured Variational Families](https://mlsanity.com/paper/2401.10989/) (2024)
- [Self-Supervised Inference in State-Space Models](https://mlsanity.com/paper/2107.13349/) (2021)
- [Sampling-Free Probabilistic Deep State-Space Models](https://mlsanity.com/paper/2309.08256/) (2023)
- [Dynamical modeling of nonlinear latent factors in multiscale neural activity with real-time inference](https://mlsanity.com/paper/2512.12462/) (2025)
- [Inferring Inference](https://mlsanity.com/paper/2310.03186/) (2023)
- [Nonparametric Filtering, Estimation and Classification using Neural Jump ODEs](https://mlsanity.com/paper/2412.03271/) (2024)
- [Large-scale Score-based Variational Posterior Inference for Bayesian Deep Neural Networks](https://mlsanity.com/paper/2602.05873/) (2026)
- [Bubblewrap: Online tiling and real-time flow prediction on neural manifolds](https://mlsanity.com/paper/2108.13941/) (2021)
- [Deep Direct Discriminative Decoders for High-dimensional Time-series Data Analysis](https://mlsanity.com/paper/2205.10947/) (2022)
- [Latent Dynamical Implicit Diffusion Processes](https://mlsanity.com/paper/2306.07077/) (2023)
- [Deep Gaussian Markov Random Fields for Graph-Structured Dynamical Systems](https://mlsanity.com/paper/2306.08445/) (2023)
- [Speech representation learning: Learning bidirectional encoders with single-view, multi-view, and multi-task methods](https://mlsanity.com/paper/2308.00129/) (2023)
- [Stochastic Deep Learning: A Probabilistic Framework for Modeling Uncertainty in Structured Temporal Data](https://mlsanity.com/paper/2601.05227/) (2026)
- [SGD for Variational Inference: Tackling Unbounded Variance via Preconditioning and Dynamic Batching](https://mlsanity.com/paper/2605.07531/) (2026)

## Abstract (arXiv)

Latent variable time-series models are among the most heavily used tools from machine learning and applied statistics. These models have the advantage of learning latent structure both from noisy observations and from the temporal ordering in the data, where it is assumed that meaningful correlation structure exists across time. A few highly-structured models, such as the linear dynamical system with linear-Gaussian observations, have closed-form inference procedures (e.g. the Kalman Filter), but this case is an exception to the general rule that exact posterior inference in more complex generative models is intractable. Consequently, much work in time-series modeling focuses on approximate inference procedures for one particular class of models. Here, we extend recent developments in stochastic variational inference to develop a `black-box' approximate inference technique for latent variable models with latent dynamical structure. We propose a structured Gaussian variational approximate posterior that carries the same intuition as the standard Kalman filter-smoother but, importantly, permits us to use the same inference approach to approximate the posterior of much more general, nonlinear latent variable generative models. We show that our approach recovers accurate estimates in the case of basic models with closed-form posteriors, and more interestingly performs well in comparison to variational approaches that were designed in a bespoke fashion for specific non-conjugate models.