# Boosting Variational Inference

arXiv: https://arxiv.org/abs/1611.05559 · HTML: https://mlsanity.com/paper/1611.05559/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Nov 2016.

## Built on by (most cited first, 22 of 22)

- [Variational Refinement for Importance Sampling Using the Forward Kullback-Leibler Divergence](https://mlsanity.com/paper/2106.15980/) (2021)
- [A Unified Perspective on Natural Gradient Variational Inference with Gaussian Mixture Models](https://mlsanity.com/paper/2209.11533/) (2022)
- [Multiple Importance Sampling ELBO and Deep Ensembles of Variational Approximations](https://mlsanity.com/paper/2202.10951/) (2022)
- [MixFlows: principled variational inference via mixed flows](https://mlsanity.com/paper/2205.07475/) (2022)
- [EigenVI: score-based variational inference with orthogonal function expansions](https://mlsanity.com/paper/2410.24054/) (2024)
- [The computational asymptotics of Gaussian variational inference and the Laplace approximation](https://mlsanity.com/paper/2104.05886/) (2021)
- [Learning from demonstration using products of experts: applications to manipulation and task prioritization](https://mlsanity.com/paper/2010.03505/) (2020)
- [End-To-End Learning of Gaussian Mixture Priors for Diffusion Sampler](https://mlsanity.com/paper/2503.00524/) (2025)
- [Boosted GFlowNets: Improving Exploration via Sequential Learning](https://mlsanity.com/paper/2511.09677/) (2025)
- [Interpolating between sampling and variational inference with infinite stochastic mixtures](https://mlsanity.com/paper/2110.09618/) (2021)
- [Fisher meets Feynman: score-based variational inference with a product of experts](https://mlsanity.com/paper/2510.21598/) (2025)
- [VFDS: Variational Foresight Dynamic Selection in Bayesian Neural Networks for Efficient Human Activity Recognition](https://mlsanity.com/paper/2204.00130/) (2022)
- [Enhancing Dynamical System Modeling through Interpretable Machine Learning Augmentations: A Case Study in Cathodic Electrophoretic Deposition](https://mlsanity.com/paper/2401.08414/) (2024)
- [Quantification of Uncertainties in Probabilistic Deep Neural Network by Implementing Boosting of Variational Inference](https://mlsanity.com/paper/2503.13909/) (2025)
- [Asymptotically exact variational flows via involutive MCMC kernels](https://mlsanity.com/paper/2506.02162/) (2025)
- [How to Approximate Inference with Subtractive Mixture Models](https://mlsanity.com/paper/2604.16714/) (2026)
- [Boltzmann Tuning of Generative Models](https://mlsanity.com/paper/2104.05252/) (2021)
- [Boosting Variational Inference With Locally Adaptive Step-Sizes](https://mlsanity.com/paper/2105.09240/) (2021)
- [Robust scalable initialization for Bayesian variational inference with multi-modal Laplace approximations](https://mlsanity.com/paper/2307.06424/) (2023)
- [Learning variational autoencoders via MCMC speed measures](https://mlsanity.com/paper/2308.13731/) (2023)
- [Measuring the right thing: justifying metrics in AI impact assessments](https://mlsanity.com/paper/2504.05007/) (2025)
- [Stepwise Variational Inference with Vine Copulas](https://mlsanity.com/paper/2603.22959/) (2026)

## Abstract (arXiv)

Variational inference (VI) provides fast approximations of a Bayesian posterior in part because it formulates posterior approximation as an optimization problem: to find the closest distribution to the exact posterior over some family of distributions. For practical reasons, the family of distributions in VI is usually constrained so that it does not include the exact posterior, even as a limit point. Thus, no matter how long VI is run, the resulting approximation will not approach the exact posterior. We propose to instead consider a more flexible approximating family consisting of all possible finite mixtures of a parametric base distribution (e.g., Gaussian). For efficient inference, we borrow ideas from gradient boosting to develop an algorithm we call boosting variational inference (BVI). BVI iteratively improves the current approximation by mixing it with a new component from the base distribution family and thereby yields progressively more accurate posterior approximations as more computing time is spent. Unlike a number of common VI variants including mean-field VI, BVI is able to capture multimodality, general posterior covariance, and nonstandard posterior shapes.