# Structured Stochastic Variational Inference

arXiv: https://arxiv.org/abs/1404.4114 · HTML: https://mlsanity.com/paper/1404.4114/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Apr 2014.

## Built on by (most cited first, 22 of 22)

- [Bayesian Learning via Neural Schr"odinger-F"ollmer Flows](https://mlsanity.com/paper/2111.10510/) (2021)
- [Sparse Inducing Points in Deep Gaussian Processes: Enhancing Modeling with Denoising Diffusion Variational Inference](https://mlsanity.com/paper/2407.17033/) (2024)
- [Structured Stochastic Gradient MCMC](https://mlsanity.com/paper/2107.09028/) (2021)
- [Automatic variational inference with cascading flows](https://mlsanity.com/paper/2102.04801/) (2021)
- [Towards Identifiability of Hierarchical Temporal Causal Representation Learning](https://mlsanity.com/paper/2510.18310/) (2025)
- [Batch, match, and patch: low-rank approximations for score-based variational inference](https://mlsanity.com/paper/2410.22292/) (2024)
- [ADAVI: Automatic Dual Amortized Variational Inference Applied To Pyramidal Bayesian Models](https://mlsanity.com/paper/2106.12248/) (2021)
- [Federated Variational Inference Methods for Structured Latent Variable Models](https://mlsanity.com/paper/2302.03314/) (2023)
- [Variational Learning of Gaussian Process Latent Variable Models through Stochastic Gradient Annealed Importance Sampling](https://mlsanity.com/paper/2408.06710/) (2024)
- [PAVI: Plate-Amortized Variational Inference](https://mlsanity.com/paper/2206.05111/) (2022)
- [Probabilistic Curriculum Learning for Goal-Based Reinforcement Learning](https://mlsanity.com/paper/2504.01459/) (2025)
- [Relay Variational Inference: A Method for Accelerated Encoderless VI](https://mlsanity.com/paper/2110.13422/) (2021)
- [Structured Recognition for Generative Models with Explaining Away](https://mlsanity.com/paper/2209.05212/) (2022)
- [DBGDGM: Dynamic Brain Graph Deep Generative Model](https://mlsanity.com/paper/2301.11408/) (2023)
- [Parameter Estimation in DAGs from Incomplete Data via Optimal Transport](https://mlsanity.com/paper/2305.15927/) (2023)
- [PAVI: Plate-Amortized Variational Inference](https://mlsanity.com/paper/2308.16022/) (2023)
- [Flexible Bayesian Last Layer Models Using Implicit Priors and Diffusion Posterior Sampling](https://mlsanity.com/paper/2408.03746/) (2024)
- [Decentralised Variational Inference Frameworks for Multi-object Tracking on Sensor Networks: Additional Notes](https://mlsanity.com/paper/2408.13689/) (2024)
- [Scalable Mean-Field Variational Inference via Preconditioned Primal-Dual Optimization](https://mlsanity.com/paper/2602.07632/) (2026)
- [A derivation of variational message passing (VMP) for latent Dirichlet allocation (LDA)](https://mlsanity.com/paper/2111.01480/) (2021)
- [Structured Nonparametric Variational Inference for Dependent Latent Modeling](https://mlsanity.com/paper/2606.15458/) (2026)
- [Amortized Structured Stochastic Variational Inference for Gaussian Process Latent Variable Models](https://mlsanity.com/paper/2610.03647/) (2026)

## Abstract (arXiv)

Stochastic variational inference makes it possible to approximate posterior distributions induced by large datasets quickly using stochastic optimization. The algorithm relies on the use of fully factorized variational distributions. However, this "mean-field" independence approximation limits the fidelity of the posterior approximation, and introduces local optima. We show how to relax the mean-field approximation to allow arbitrary dependencies between global parameters and local hidden variables, producing better parameter estimates by reducing bias, sensitivity to local optima, and sensitivity to hyperparameters.