# Note on the bias and variance of variational inference

arXiv: https://arxiv.org/abs/1906.03708 · HTML: https://mlsanity.com/paper/1906.03708/

## Key facts

- Cited by 4 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 9 Jun 2019.

## Built on by (most cited first, 4 of 4)

- [LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Models](https://mlsanity.com/paper/2505.19223/) (2025)
- [Mind the Gap when Conditioning Amortised Inference in Sequential Latent-Variable Models](https://mlsanity.com/paper/2101.07046/) (2021)
- [Loss function based second-order Jensen inequality and its application to particle variational inference](https://mlsanity.com/paper/2106.05010/) (2021)
- [Uphill Roads to Variational Tightness: Monotonicity and Monte Carlo Objectives](https://mlsanity.com/paper/2201.10989/) (2022)

## Abstract (arXiv)

In this note, we study the relationship between the variational gap and the variance of the (log) likelihood ratio. We show that the gap can be upper bounded by some form of dispersion measure of the likelihood ratio, which suggests the bias of variational inference can be reduced by making the distribution of the likelihood ratio more concentrated, such as via averaging and variance reduction.