# f-Divergence Variational Inference

arXiv: https://arxiv.org/abs/2009.13093 · HTML: https://mlsanity.com/paper/2009.13093/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Sep 2020.

## Built on by (most cited first, 28 of 28)

- [GFlowNets and variational inference](https://mlsanity.com/paper/2210.00580/) (2022)
- [Beyond ELBOs: A Large-Scale Evaluation of Variational Methods for Sampling](https://mlsanity.com/paper/2406.07423/) (2024)
- [Variational Wasserstein gradient flow](https://mlsanity.com/paper/2112.02424/) (2021)
- [Variational methods for simulation-based inference](https://mlsanity.com/paper/2203.04176/) (2022)
- [One-step Diffusion Models with f-Divergence Distribution Matching](https://mlsanity.com/paper/2502.15681/) (2025)
- [Challenges and Opportunities in High-dimensional Variational Inference](https://mlsanity.com/paper/2103.01085/) (2021)
- [Sliced Wasserstein Variational Inference](https://mlsanity.com/paper/2207.13177/) (2022)
- [Variational Inference MPC using Tsallis Divergence](https://mlsanity.com/paper/2104.00241/) (2021)
- [Bayesian brains and the R'enyi divergence](https://mlsanity.com/paper/2107.05438/) (2021)
- [Markov Chain Score Ascent: A Unifying Framework of Variational Inference with Markovian Gradients](https://mlsanity.com/paper/2206.06295/) (2022)
- [Jensen-Shannon Divergence Based Novel Loss Functions for Bayesian Neural Networks](https://mlsanity.com/paper/2209.11366/) (2022)
- [End-To-End Learning of Gaussian Mixture Priors for Diffusion Sampler](https://mlsanity.com/paper/2503.00524/) (2025)
- [AdAEM: An Adaptively and Automated Extensible Measurement of LLMs' Value Difference](https://mlsanity.com/paper/2505.13531/) (2025)
- [Greedy Bayesian Posterior Approximation with Deep Ensembles](https://mlsanity.com/paper/2105.14275/) (2021)
- [Variational Inference with Holder Bounds](https://mlsanity.com/paper/2111.02947/) (2021)
- [Optimal Regret Is Achievable with Bounded Approximate Inference Error: An Enhanced Bayesian Upper Confidence Bound Framework](https://mlsanity.com/paper/2201.12955/) (2022)
- [Improving Variational Autoencoder Estimation from Incomplete Data with Mixture Variational Families](https://mlsanity.com/paper/2403.03069/) (2024)
- [Differentiable Annealed Importance Sampling Minimizes The Symmetrized Kullback-Leibler Divergence Between Initial and Target Distribution](https://mlsanity.com/paper/2405.14840/) (2024)
- [Generalized Munchausen Reinforcement Learning using Tsallis KL Divergence](https://mlsanity.com/paper/2301.11476/) (2023)
- [Parameter Estimation in DAGs from Incomplete Data via Optimal Transport](https://mlsanity.com/paper/2305.15927/) (2023)
- [R'enyi Neural Processes](https://mlsanity.com/paper/2405.15991/) (2024)
- [SoftCVI: Contrastive variational inference with self-generated soft labels](https://mlsanity.com/paper/2407.15687/) (2024)
- [Censor Dependent Variational Inference](https://mlsanity.com/paper/2502.09591/) (2025)
- [Accelerating Hamiltonian Monte Carlo for Bayesian Inference in Neural Networks and Neural Operators](https://mlsanity.com/paper/2507.14652/) (2025)
- [Constrained Stein Variational Gradient Descent for Robot Perception, Planning, and Identification](https://mlsanity.com/paper/2506.00589/) (2025)
- [Spatio-temporal probabilistic forecast using MMAF-guided learning](https://mlsanity.com/paper/2603.15055/) (2026)
- [Bayesian 3D Steerable CNNs: Enabling Equivariance and Uncertainty Quantification Simultaneously](https://mlsanity.com/paper/2606.15479/) (2026)
- [Beyond Global Divergences: A Local-Mass Perspective on Bayesian Inference](https://mlsanity.com/paper/2606.27090/) (2026)

## Abstract (arXiv)

This paper introduces the f-divergence variational inference (f-VI) that generalizes variational inference to all f-divergences. Initiated from minimizing a crafty surrogate f-divergence that shares the statistical consistency with the f-divergence, the f-VI framework not only unifies a number of existing VI methods, e.g. Kullback-Leibler VI, R'enyi's α-VI, and chi-VI, but offers a standardized toolkit for VI subject to arbitrary divergences from f-divergence family. A general f-variational bound is derived and provides a sandwich estimate of marginal likelihood (or evidence). The development of the f-VI unfolds with a stochastic optimization scheme that utilizes the reparameterization trick, importance weighting and Monte Carlo approximation; a mean-field approximation scheme that generalizes the well-known coordinate ascent variational inference (CAVI) is also proposed for f-VI. Empirical examples, including variational autoencoders and Bayesian neural networks, are provided to demonstrate the effectiveness and the wide applicability of f-VI.