mlsanityData to 7 Oct 2026

f-Divergence Variational Inference

arXiv 2009.13093 ↗First posted 28 Sep 2020cs.LG cs.IT math.ITcited by 28

Not on the map. Why?

Abstract (arXiv)

This paper introduces the f-divergence variational inference (f-VI) that generalizes variational inference to all f-divergences. Initiated from minimizing a crafty surrogate f-divergence that shares the statistical consistency with the f-divergence, the f-VI framework not only unifies a number of existing VI methods, e.g. Kullback-Leibler VI, R'enyi's α-VI, and chi-VI, but offers a standardized toolkit for VI subject to arbitrary divergences from f-divergence family. A general f-variational bound is derived and provides a sandwich estimate of marginal likelihood (or evidence). The development of the f-VI unfolds with a stochastic optimization scheme that utilizes the reparameterization trick, importance weighting and Monte Carlo approximation; a mean-field approximation scheme that generalizes the well-known coordinate ascent variational inference (CAVI) is also proposed for f-VI. Empirical examples, including variational autoencoders and Bayesian neural networks, are provided to demonstrate the effectiveness and the wide applicability of f-VI.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Sep 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 6; 2023: 2; 2024: 5; 2025: 6; 2026: 3.

28 papers.

Sort:

By area:

  1. 2022GFlowNets and variational inferencecited by 103
  2. 2024Beyond ELBOs: A Large-Scale Evaluation of Variational Methods for Samplingcited by 55
  3. 2021Variational Wasserstein gradient flowcited by 53
  4. 2022Variational methods for simulation-based inferencecited by 43
  5. 2025One-step Diffusion Models with f-Divergence Distribution Matchingcited by 43
  6. 2021Challenges and Opportunities in High-dimensional Variational Inferencecited by 35
  7. 2022Sliced Wasserstein Variational Inferencecited by 17
  8. 2021Variational Inference MPC using Tsallis Divergencecited by 11
  9. 2021Bayesian brains and the R'enyi divergencecited by 6
  10. 2022Markov Chain Score Ascent: A Unifying Framework of Variational Inference with Markovian Gradientscited by 6
  11. 2022Jensen-Shannon Divergence Based Novel Loss Functions for Bayesian Neural Networkscited by 6
  12. 2025End-To-End Learning of Gaussian Mixture Priors for Diffusion Samplercited by 4
16 more
  1. 2025AdAEM: An Adaptively and Automated Extensible Measurement of LLMs' Value Differencecited by 4
  2. 2021Greedy Bayesian Posterior Approximation with Deep Ensemblescited by 3
  3. 2021Variational Inference with Holder Boundscited by 3
  4. 2022Optimal Regret Is Achievable with Bounded Approximate Inference Error: An Enhanced Bayesian Upper Confidence Bound Frameworkcited by 2
  5. 2024Improving Variational Autoencoder Estimation from Incomplete Data with Mixture Variational Familiescited by 2
  6. 2024Differentiable Annealed Importance Sampling Minimizes The Symmetrized Kullback-Leibler Divergence Between Initial and Target Distributioncited by 2
  7. 2023Generalized Munchausen Reinforcement Learning using Tsallis KL Divergencecited by 1
  8. 2023Parameter Estimation in DAGs from Incomplete Data via Optimal Transportcited by 1
  9. 2024R'enyi Neural Processescited by 1
  10. 2024SoftCVI: Contrastive variational inference with self-generated soft labelscited by 1
  11. 2025Censor Dependent Variational Inferencecited by 1
  12. 2025Accelerating Hamiltonian Monte Carlo for Bayesian Inference in Neural Networks and Neural Operatorscited by 1
  13. 2025Constrained Stein Variational Gradient Descent for Robot Perception, Planning, and Identificationcited by 0
  14. 2026Spatio-temporal probabilistic forecast using MMAF-guided learningcited by 0
  15. 2026Bayesian 3D Steerable CNNs: Enabling Equivariance and Uncertainty Quantification Simultaneouslycited by 0
  16. 2026Beyond Global Divergences: A Local-Mass Perspective on Bayesian Inferencecited by 0