# Mixture weights optimisation for Alpha-Divergence Variational Inference

arXiv: https://arxiv.org/abs/2106.05114 · HTML: https://mlsanity.com/paper/2106.05114/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 43 references that are in the map.
- First posted on arXiv 9 Jun 2021.

## Builds on (most cited first, 13 of 13)

- [Convex Optimization: Algorithms and Complexity](https://mlsanity.com/paper/1405.4980/) (2014)
- [R'enyi Divergence Variational Inference](https://mlsanity.com/paper/1602.02311/) (2016)
- [Challenges and Opportunities in High-dimensional Variational Inference](https://mlsanity.com/paper/2103.01085/) (2021)
- [Provable Smoothness Guarantees for Black-Box Variational Inference](https://mlsanity.com/paper/1901.08431/) (2019)
- [Variational Inference with Tail-adaptive f-Divergence](https://mlsanity.com/paper/1810.11943/) (2018)
- [Provable Gradient Variance Guarantees for Black-Box Variational Inference](https://mlsanity.com/paper/1906.08241/) (2019)
- [On the Difficulty of Unbiased Alpha Divergence Minimization](https://mlsanity.com/paper/2010.09541/) (2020)
- [Consistency of Variational Bayes Inference for Estimation and Model Selection in Mixtures](https://mlsanity.com/paper/1805.05054/) (2018)
- [Empirical Evaluation of Biased Methods for Alpha Divergence Minimization](https://mlsanity.com/paper/2105.06587/) (2021)
- [Perturbative Black Box Variational Inference](https://mlsanity.com/paper/1709.07433/) (2017)
- [Variational Inference via chi-Upper Bound Minimization](https://mlsanity.com/paper/1611.00328/) (2016)
- [Neural Variational Inference and Learning in Undirected Graphical Models](https://mlsanity.com/paper/1711.02679/) (2017)
- [Meta-Learning Divergences of Variational Inference](https://mlsanity.com/paper/2007.02912/) (2020)

## Built on by (most cited first, 6 of 6)

- [Variational inference via Wasserstein gradient flows](https://mlsanity.com/paper/2205.15902/) (2022)
- [Sampling via Gradient Flows in the Space of Probability Measures](https://mlsanity.com/paper/2310.03597/) (2023)
- [Alpha-divergence Variational Inference Meets Importance Weighted Auto-Encoders: Methodology and Asymptotics](https://mlsanity.com/paper/2210.06226/) (2022)
- [Optimal Regret Is Achievable with Bounded Approximate Inference Error: An Enhanced Bayesian Upper Confidence Bound Framework](https://mlsanity.com/paper/2201.12955/) (2022)
- [Variational Online Mirror Descent for Robust Learning in Schr"odinger Bridge](https://mlsanity.com/paper/2504.02618/) (2025)
- [Even More Guarantees for Variational Inference in the Presence of Symmetries](https://mlsanity.com/paper/2604.21407/) (2026)

## Abstract (arXiv)

This paper focuses on α-divergence minimisation methods for Variational Inference. More precisely, we are interested in algorithms optimising the mixture weights of any given mixture model, without any information on the underlying distribution of its mixture components parameters. The Power Descent, defined for all α neq 1, is one such algorithm and we establish in our work the full proof of its convergence towards the optimal mixture weights when α <1. Since the α-divergence recovers the widely-used forward Kullback-Leibler when α → 1, we then extend the Power Descent to the case α = 1 and show that we obtain an Entropic Mirror Descent. This leads us to investigate the link between Power Descent and Entropic Mirror Descent: first-order approximations allow us to introduce the Renyi Descent, a novel algorithm for which we prove an O(1/N) convergence rate. Lastly, we compare numerically the behavior of the unbiased Power Descent and of the biased Renyi Descent and we discuss the potential advantages of one algorithm over the other.