mlsanityData to 14 Oct 2026

R'enyi Divergence Variational Inference

arXiv 1602.02311 ↗First posted 6 Feb 2016stat.ML cs.LGcited by 104

Not on the map. Why?

Abstract (arXiv)

This paper introduces the variational R'enyi bound (VR) that extends traditional variational inference to R'enyi's alpha-divergences. This new family of variational methods unifies a number of existing approaches, and enables a smooth interpolation from the evidence lower-bound to the log (marginal) likelihood that is controlled by the value of alpha that parametrises the divergence. The reparameterization trick, Monte Carlo approximation and stochastic optimisation methods are deployed to obtain a tractable and unified framework for optimisation. We further consider negative alpha values and propose a novel variational inference method as a new special case in the proposed framework. Experiments on Bayesian neural networks and variational auto-encoders demonstrate the wide applicability of the VR bound.

Key facts. Cited by 104 papers in the map (data through 14 Oct 2026). First posted on arXiv 6 Feb 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 19; 2022: 20; 2023: 13; 2024: 13; 2025: 20; 2026: 19.

104 papers.

Sort:

By area:

  1. 2023Generative Time Series Forecasting with Diffusion, Denoise, and Disentanglementcited by 90
  2. 2024Beyond ELBOs: A Large-Scale Evaluation of Variational Methods for Samplingcited by 56
  3. 2022Variational methods for simulation-based inferencecited by 43
  4. 2021Challenges and Opportunities in High-dimensional Variational Inferencecited by 35
  5. 2021InfoLM: A New Metric to Evaluate Summarization & Data2Text Generationcited by 33
  6. 2021Variational Refinement for Importance Sampling Using the Forward Kullback-Leibler Divergencecited by 29
  7. 2023Transport meets Variational Inference: Controlled Monte Carlo Diffusionscited by 21
  8. 2022Sliced Wasserstein Variational Inferencecited by 18
  9. 2023On the Convergence of Coordinate Ascent Variational Inferencecited by 18
  10. 2023t3-Variational Autoencoder: Learning Heavy-tailed Data with Student's t and Power Divergencecited by 18
  11. 2025Flow Density Control: Generative Optimization Beyond Entropy-Regularized Fine-Tuningcited by 17
  12. 2022Amortized Variational Inference: A Systematic Reviewcited by 16
38 more
  1. 2025Understanding Multimodal LLMs Under Distribution Shifts: An Information-Theoretic Approachcited by 16
  2. 2022Alpha-divergence Variational Inference Meets Importance Weighted Auto-Encoders: Methodology and Asymptoticscited by 15
  3. 2021Mind the Gap when Conditioning Amortised Inference in Sequential Latent-Variable Modelscited by 14
  4. 2022Multiple Importance Sampling ELBO and Deep Ensembles of Variational Approximationscited by 12
  5. 2021Variational Inference MPC using Tsallis Divergencecited by 11
  6. 2021PriorVAE: Encoding spatial priors with VAEs for small-area estimationcited by 11
  7. 2022Fat-Tailed Variational Inference with Anisotropic Tail Adaptive Flowscited by 11
  8. 2024Variational Inference for Uncertainty Quantification: an Analysis of Trade-offscited by 11
  9. 2024Beyond Squared Error: Exploring Loss Design for Enhanced Training of Generative Flow Networkscited by 11
  10. 2021Empirical Evaluation of Biased Methods for Alpha Divergence Minimizationcited by 10
  11. 2025Loss Functions and Operators Generated by f-Divergencescited by 10
  12. 2023The Representation Jensen-Shannon Divergencecited by 9
  13. 2024Variational Inference in Location-Scale Families: Exact Recovery of the Mean and Correlation Matrixcited by 9
  14. 2021Spiking Generative Adversarial Networks With a Neural Network Discriminator: Local Training, Bayesian Models, and Continual Meta-Learningcited by 8
  15. 2022RenyiCL: Contrastive Representation Learning with Skew Renyi Divergencecited by 8
  16. 2023Bridging the Gap Between Variational Inference and Wasserstein Gradient Flowscited by 8
  17. 2021Pseudo-Spherical Contrastive Divergencecited by 7
  18. 2022Function-space regularized R'enyi divergencescited by 7
  19. 2021Mixture weights optimisation for Alpha-Divergence Variational Inferencecited by 6
  20. 2021Bayesian brains and the R'enyi divergencecited by 6
  21. 2022Robust PACm: Training Ensemble Models Under Misspecification and Outlierscited by 6
  22. 2022Markov Chain Score Ascent: A Unifying Framework of Variational Inference with Markovian Gradientscited by 6
  23. 2022Bounding Evidence and Estimating Log-Likelihood in VAEcited by 6
  24. 2022Jensen-Shannon Divergence Based Novel Loss Functions for Bayesian Neural Networkscited by 6
  25. 2024On Divergence Measures for Training GFlowNetscited by 6
  26. 2021Monotonic Alpha-divergence Minimisation for Variational Inferencecited by 5
  27. 2022Variational Open-Domain Question Answeringcited by 5
  28. 2023Machine Learning and the Future of Bayesian Computationcited by 4
  29. 2024Alpha-VI DeepONet: A prior-robust variational Bayesian approach for enhancing DeepONets with uncertainty quantificationcited by 4
  30. 2025End-To-End Learning of Gaussian Mixture Priors for Diffusion Samplercited by 4
  31. 2021An Easy to Interpret Diagnostic for Approximate Inference: Symmetric Divergence Over Simulationscited by 3
  32. 2021Variational Inference with Holder Boundscited by 3
  33. 2021Computing Divergences between Discrete Decomposable Modelscited by 3
  34. 2022Renyi Fair Information Bottleneck for Image Classificationcited by 3
  35. 2023Adaptive importance sampling for heavy-tailed distributions via α-divergence minimizationcited by 3
  36. 2024Risk-sensitive control as inference with R'enyi divergencecited by 3
  37. 2025Generalized Guarantees for Variational Inference in the Presence of Even and Elliptical Symmetrycited by 3
  38. 2021Toward an Idiomatic Framework for Cognitive Roboticscited by 2

50 most cited of 104 All 104 (JSON)