mlsanityData to 14 Oct 2026

Empirical Evaluation of Biased Methods for Alpha Divergence Minimization

arXiv 2105.06587 ↗First posted 13 May 2021cs.LG stat.MLcited by 10

Not on the map. Why?

Abstract (arXiv)

In this paper we empirically evaluate biased methods for alpha-divergence minimization. In particular, we focus on how the bias affects the final solutions found, and how this depends on the dimensionality of the problem. We find that (i) solutions returned by these methods appear to be strongly biased towards minimizers of the traditional "exclusive" KL-divergence, KL(q||p), and (ii) in high dimensions, an impractically large amount of computation is needed to mitigate this bias and obtain solutions that actually minimize the alpha-divergence of interest.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 13 of its 19 references that are in the map. First posted on arXiv 13 May 2021.

Builds on

13 of 19 references.

References · 13

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2013Auto-Encoding Variational Bayescited by 4,053
  3. 2015Importance Weighted Autoencoderscited by 456
  4. 2016R'enyi Divergence Variational Inferencecited by 104
  5. 2017Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inferencecited by 98
  6. 2017Filtering Variational Objectivescited by 91
  7. 2014Reweighted Wake-Sleepcited by 66
  8. 2018Importance Weighting and Variational Inferencecited by 55
  9. 2020Markovian Score Climbing: Variational Inference with KL(p||q)cited by 46
  10. 2018Doubly Reparameterized Gradient Estimators for Monte Carlo Objectivescited by 41
  11. 2016Edward: A library for probabilistic modeling, inference, and criticismcited by 28
  12. 2020On the Difficulty of Unbiased Alpha Divergence Minimizationcited by 21
1 more
  1. 2017Neural Variational Inference and Learning in Undirected Graphical Modelscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 5; 2023: 0; 2024: 3; 2025: 0; 2026: 1.

10 papers.

  1. 2022Gradients should stay on Path: Better Estimators of the Reverse- and Forward KL Divergence for Normalizing Flowscited by 23
  2. 2022Alpha-divergence Variational Inference Meets Importance Weighted Auto-Encoders: Methodology and Asymptoticscited by 15
  3. 2022Path-Gradient Estimators for Continuous Normalizing Flowscited by 11
  4. 2021Mixture weights optimisation for Alpha-Divergence Variational Inferencecited by 6
  5. 2022Markov Chain Score Ascent: A Unifying Framework of Variational Inference with Markovian Gradientscited by 6
  6. 2024Fast and Unified Path Gradient Estimators for Normalizing Flowscited by 5
  7. 2024Disentangling impact of capacity, objective, batchsize, estimators, and step-size on flow VIcited by 5
  8. 2022Variational Inference with Locally Enhanced Bounds for Hierarchical Modelscited by 4
  9. 2024Variational Learning of Gaussian Process Latent Variable Models through Stochastic Gradient Annealed Importance Samplingcited by 4
  10. 2026Large-scale empirical tuning and comparison of default optimizers for variational inferencecited by 0