# Overdispersed Black-Box Variational Inference

arXiv: https://arxiv.org/abs/1603.01140 · HTML: https://mlsanity.com/paper/1603.01140/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Mar 2016.

## Built on by (most cited first, 8 of 8)

- [Provable convergence guarantees for black-box variational inference](https://mlsanity.com/paper/2306.03638/) (2023)
- [Amortized Variational Inference: A Systematic Review](https://mlsanity.com/paper/2209.10888/) (2022)
- [Generalized Doubly Reparameterized Gradient Estimators](https://mlsanity.com/paper/2101.11046/) (2021)
- [A unified view of likelihood ratio and reparameterization gradients](https://mlsanity.com/paper/2105.14900/) (2021)
- [Coupled Gradient Estimators for Discrete Latent Variables](https://mlsanity.com/paper/2106.08056/) (2021)
- [Improved Variational Bayesian Phylogenetic Inference using Mixtures](https://mlsanity.com/paper/2310.00941/) (2023)
- [Understanding and mitigating difficulties in posterior predictive evaluation](https://mlsanity.com/paper/2405.19747/) (2024)
- [You Only Accept Samples Once: Fast, Self-Correcting Stochastic Variational Inference](https://mlsanity.com/paper/2406.02838/) (2024)

## Abstract (arXiv)

We introduce overdispersed black-box variational inference, a method to reduce the variance of the Monte Carlo estimator of the gradient in black-box variational inference. Instead of taking samples from the variational distribution, we use importance sampling to take samples from an overdispersed distribution in the same exponential family as the variational approximation. Our approach is general since it can be readily applied to any exponential family distribution, which is the typical choice for the variational approximation. We run experiments on two non-conjugate probabilistic models to show that our method effectively reduces the variance, and the overhead introduced by the computation of the proposal parameters and the importance weights is negligible. We find that our overdispersed importance sampling scheme provides lower variance than black-box variational inference, even when the latter uses twice the number of samples. This results in faster convergence of the black-box inference procedure.