# Proximity Variational Inference

arXiv: https://arxiv.org/abs/1705.08931 · HTML: https://mlsanity.com/paper/1705.08931/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 May 2017.

## Built on by (most cited first, 6 of 6)

- [On the Convergence of Black-Box Variational Inference](https://mlsanity.com/paper/2305.15349/) (2023)
- [Variational Gibbs Inference for Statistical Model Estimation from Incomplete Data](https://mlsanity.com/paper/2111.13180/) (2021)
- [Provably Scalable Black-Box Variational Inference with Structured Variational Families](https://mlsanity.com/paper/2401.10989/) (2024)
- [Evaluating Bayesian deep learning for radio galaxy classification](https://mlsanity.com/paper/2405.18351/) (2024)
- [VICatMix: variational Bayesian clustering and variable selection for discrete biomedical data](https://mlsanity.com/paper/2406.16227/) (2024)
- [Globally Convergent Variational Inference](https://mlsanity.com/paper/2501.08201/) (2025)

## Abstract (arXiv)

Variational inference is a powerful approach for approximate posterior inference. However, it is sensitive to initialization and can be subject to poor local optima. In this paper, we develop proximity variational inference (PVI). PVI is a new method for optimizing the variational objective that constrains subsequent iterates of the variational parameters to robustify the optimization path. Consequently, PVI is less sensitive to initialization and optimization quirks and finds better local optima. We demonstrate our method on three proximity statistics. We study PVI on a Bernoulli factor model and sigmoid belief network with both real and synthetic data and compare to deterministic annealing (Katahira et al., 2008). We highlight the flexibility of PVI by designing a proximity statistic for Bayesian deep learning models such as the variational autoencoder (Kingma and Welling, 2014; Rezende et al., 2014). Empirically, we show that PVI consistently finds better local optima and gives better predictive performance.