# Automatic structured variational inference

arXiv: https://arxiv.org/abs/2002.00643 · HTML: https://mlsanity.com/paper/2002.00643/

## Key facts

- Cited by 21 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 3 Feb 2020.

## Built on by (most cited first, 21 of 21)

- [Amortized Variational Inference: When and Why?](https://mlsanity.com/paper/2307.11018/) (2023)
- [Amortized Variational Inference for Simple Hierarchical Models](https://mlsanity.com/paper/2111.03144/) (2021)
- [Nested Variational Inference](https://mlsanity.com/paper/2106.11302/) (2021)
- [Structured Stochastic Gradient MCMC](https://mlsanity.com/paper/2107.09028/) (2021)
- [Automatic variational inference with cascading flows](https://mlsanity.com/paper/2102.04801/) (2021)
- [Neuroscience-inspired perception-action in robotics: applying active inference for state estimation, control and self-perception](https://mlsanity.com/paper/2105.04261/) (2021)
- [Variational Inference with Locally Enhanced Bounds for Hierarchical Models](https://mlsanity.com/paper/2203.04432/) (2022)
- [Federated Variational Inference Methods for Structured Latent Variable Models](https://mlsanity.com/paper/2302.03314/) (2023)
- [Provably Scalable Black-Box Variational Inference with Structured Variational Families](https://mlsanity.com/paper/2401.10989/) (2024)
- [ADAVI: Automatic Dual Amortized Variational Inference Applied To Pyramidal Bayesian Models](https://mlsanity.com/paper/2106.12248/) (2021)
- [Embedded-model flows: Combining the inductive biases of model-free deep learning and explicit probabilistic modeling](https://mlsanity.com/paper/2110.06021/) (2021)
- [Model-Informed Flows for Bayesian Inference](https://mlsanity.com/paper/2505.24243/) (2025)
- [Flow Annealed Kalman Inversion for Gradient-Free Inference in Bayesian Inverse Problems](https://mlsanity.com/paper/2309.11490/) (2023)
- [Rethinking Variational Inference for Probabilistic Programs with Stochastic Support](https://mlsanity.com/paper/2311.00594/) (2023)
- [Bayesian Experimental Design for Model Discrepancy Calibration: An Auto-Differentiable Ensemble Kalman Inversion Approach](https://mlsanity.com/paper/2504.20319/) (2025)
- [PAVI: Plate-Amortized Variational Inference](https://mlsanity.com/paper/2206.05111/) (2022)
- [Computing with Categories in Machine Learning](https://mlsanity.com/paper/2303.04156/) (2023)
- [Parameter Estimation in DAGs from Incomplete Data via Optimal Transport](https://mlsanity.com/paper/2305.15927/) (2023)
- [PAVI: Plate-Amortized Variational Inference](https://mlsanity.com/paper/2308.16022/) (2023)
- [Understanding and mitigating difficulties in posterior predictive evaluation](https://mlsanity.com/paper/2405.19747/) (2024)
- [Bayesian Experimental Design for Model Discrepancy Calibration: A Rivalry between Kullback--Leibler Divergence and Wasserstein Distance](https://mlsanity.com/paper/2601.16425/) (2026)

## Abstract (arXiv)

Stochastic variational inference offers an attractive option as a default method for differentiable probabilistic programming. However, the performance of the variational approach depends on the choice of an appropriate variational family. Here, we introduce automatic structured variational inference (ASVI), a fully automated method for constructing structured variational families, inspired by the closed-form update in conjugate Bayesian models. These convex-update families incorporate the forward pass of the input probabilistic program and can therefore capture complex statistical dependencies. Convex-update families have the same space and time complexity as the input probabilistic program and are therefore tractable for a very large family of models including both continuous and discrete variables. We validate our automatic variational method on a wide range of low- and high-dimensional inference problems. We find that ASVI provides a clear improvement in performance when compared with other popular approaches such as the mean-field approach and inverse autoregressive flows. We provide an open source implementation of ASVI in TensorFlow Probability.