# Variational Boosting: Iteratively Refining Posterior Approximations

arXiv: https://arxiv.org/abs/1611.06585 · HTML: https://mlsanity.com/paper/1611.06585/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Nov 2016.

## Built on by (most cited first, 30 of 37)

- [Challenges and Opportunities in High-dimensional Variational Inference](https://mlsanity.com/paper/2103.01085/) (2021)
- [Is MC Dropout Bayesian?](https://mlsanity.com/paper/2110.04286/) (2021)
- [Variational Refinement for Importance Sampling Using the Forward Kullback-Leibler Divergence](https://mlsanity.com/paper/2106.15980/) (2021)
- [A Unified Perspective on Natural Gradient Variational Inference with Gaussian Mixture Models](https://mlsanity.com/paper/2209.11533/) (2022)
- [Riemannian Laplace approximations for Bayesian neural networks](https://mlsanity.com/paper/2306.07158/) (2023)
- [Posterior Refinement Improves Sample Efficiency in Bayesian Neural Networks](https://mlsanity.com/paper/2205.10041/) (2022)
- [A theoretical perspective on mode collapse in variational inference](https://mlsanity.com/paper/2410.13300/) (2024)
- [MixFlows: principled variational inference via mixed flows](https://mlsanity.com/paper/2205.07475/) (2022)
- [EigenVI: score-based variational inference with orthogonal function expansions](https://mlsanity.com/paper/2410.24054/) (2024)
- [Statistical Perspectives on Reliability of Artificial Intelligence Systems](https://mlsanity.com/paper/2111.05391/) (2021)
- [The computational asymptotics of Gaussian variational inference and the Laplace approximation](https://mlsanity.com/paper/2104.05886/) (2021)
- [Batch, match, and patch: low-rank approximations for score-based variational inference](https://mlsanity.com/paper/2410.22292/) (2024)
- [Statistical and Computational Trade-offs in Variational Inference: A Case Study in Inferential Model Selection](https://mlsanity.com/paper/2207.11208/) (2022)
- [Variational Marginal Particle Filters](https://mlsanity.com/paper/2109.15134/) (2021)
- [Learning from demonstration using products of experts: applications to manipulation and task prioritization](https://mlsanity.com/paper/2010.03505/) (2020)
- [End-To-End Learning of Gaussian Mixture Priors for Diffusion Sampler](https://mlsanity.com/paper/2503.00524/) (2025)
- [Boosted GFlowNets: Improving Exploration via Sequential Learning](https://mlsanity.com/paper/2511.09677/) (2025)
- [Interpolating between sampling and variational inference with infinite stochastic mixtures](https://mlsanity.com/paper/2110.09618/) (2021)
- [Variational Inference with Holder Bounds](https://mlsanity.com/paper/2111.02947/) (2021)
- [Fast post-process Bayesian inference with Variational Sparse Bayesian Quadrature](https://mlsanity.com/paper/2303.05263/) (2023)
- [Tree-based variational inference for Poisson log-normal models](https://mlsanity.com/paper/2406.17361/) (2024)
- [Fisher meets Feynman: score-based variational inference with a product of experts](https://mlsanity.com/paper/2510.21598/) (2025)
- [Cyclical Variational Bayes Monte Carlo for Efficient Multi-Modal Posterior Distributions Evaluation](https://mlsanity.com/paper/2202.11645/) (2022)
- [VFDS: Variational Foresight Dynamic Selection in Bayesian Neural Networks for Efficient Human Activity Recognition](https://mlsanity.com/paper/2204.00130/) (2022)
- [Large Skew-t Copula Models and Asymmetric Dependence in Intraday Equity Returns](https://mlsanity.com/paper/2308.05564/) (2023)
- [ELBOing Stein: Variational Bayes with Stein Mixture Inference](https://mlsanity.com/paper/2410.22948/) (2024)
- [Quantification of Uncertainties in Probabilistic Deep Neural Network by Implementing Boosting of Variational Inference](https://mlsanity.com/paper/2503.13909/) (2025)
- [Asymptotically exact variational flows via involutive MCMC kernels](https://mlsanity.com/paper/2506.02162/) (2025)
- [Walking on the Fiber: A Simple Geometric Approximation for Bayesian Neural Networks](https://mlsanity.com/paper/2512.01500/) (2025)
- [Boltzmann Tuning of Generative Models](https://mlsanity.com/paper/2104.05252/) (2021)

## Abstract (arXiv)

We propose a black-box variational inference method to approximate intractable distributions with an increasingly rich approximating class. Our method, termed variational boosting, iteratively refines an existing variational approximation by solving a sequence of optimization problems, allowing the practitioner to trade computation time for accuracy. We show how to expand the variational approximating class by incorporating additional covariance structure and by introducing new components to form a mixture. We apply variational boosting to synthetic and real statistical models, and show that resulting posterior inferences compare favorably to existing posterior approximation algorithms in both accuracy and efficiency.