Note on the bias and variance of variational inference
Not on the map. Why?
Abstract (arXiv)
In this note, we study the relationship between the variational gap and the variance of the (log) likelihood ratio. We show that the gap can be upper bounded by some form of dispersion measure of the likelihood ratio, which suggests the bias of variational inference can be reduced by making the distribution of the likelihood ratio more concentrated, such as via averaging and variance reduction.
Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Jun 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 2; 2022: 1; 2023: 0; 2024: 0; 2025: 1.
4 papers.
- 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 273
- 2021Mind the Gap when Conditioning Amortised Inference in Sequential Latent-Variable Modelscited by 14
- 2021Loss function based second-order Jensen inequality and its application to particle variational inferencecited by 3
- 2022Uphill Roads to Variational Tightness: Monotonicity and Monte Carlo Objectivescited by 3
- 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Models25 May 2025
- 2022Uphill Roads to Variational Tightness: Monotonicity and Monte Carlo Objectives26 Jan 2022
- 2021Loss function based second-order Jensen inequality and its application to particle variational inference9 Jun 2021
- 2021Mind the Gap when Conditioning Amortised Inference in Sequential Latent-Variable Models18 Jan 2021