# Black-box α-divergence Minimization

arXiv: https://arxiv.org/abs/1511.03243 · HTML: https://mlsanity.com/paper/1511.03243/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Nov 2015.

## Built on by (most cited first, 30 of 37)

- [GFlowNets and variational inference](https://mlsanity.com/paper/2210.00580/) (2022)
- [Confidence May Cheat: Self-Training on Graph Neural Networks under Distribution Shift](https://mlsanity.com/paper/2201.11349/) (2022)
- [Challenges and Opportunities in High-dimensional Variational Inference](https://mlsanity.com/paper/2103.01085/) (2021)
- [Adapting the Linearised Laplace Model Evidence for Modern Deep Learning](https://mlsanity.com/paper/2206.08900/) (2022)
- [On the Convergence of Black-Box Variational Inference](https://mlsanity.com/paper/2305.15349/) (2023)
- [Transport meets Variational Inference: Controlled Monte Carlo Diffusions](https://mlsanity.com/paper/2307.01050/) (2023)
- [Variational Inference for Uncertainty Quantification: an Analysis of Trade-offs](https://mlsanity.com/paper/2403.13748/) (2024)
- [Calib3D: Calibrating Model Preferences for Reliable 3D Scene Understanding](https://mlsanity.com/paper/2403.17010/) (2024)
- [Deep Variational Implicit Processes](https://mlsanity.com/paper/2206.06720/) (2022)
- [U4D: Uncertainty-Aware 4D World Modeling from LiDAR Sequences](https://mlsanity.com/paper/2512.02982/) (2025)
- [Markov Chain Score Ascent: A Unifying Framework of Variational Inference with Markovian Gradients](https://mlsanity.com/paper/2206.06295/) (2022)
- [Bayesian neural networks for predicting uncertainty in full-field material response](https://mlsanity.com/paper/2406.14838/) (2024)
- [Forward chi2 Divergence Based Variational Importance Sampling](https://mlsanity.com/paper/2311.02516/) (2023)
- [Variational Inference with Holder Bounds](https://mlsanity.com/paper/2111.02947/) (2021)
- [Adaptive importance sampling for heavy-tailed distributions via α-divergence minimization](https://mlsanity.com/paper/2310.16653/) (2023)
- [Hellinger Multimodal Variational Autoencoders](https://mlsanity.com/paper/2601.06572/) (2026)
- [Optimal Regret Is Achievable with Bounded Approximate Inference Error: An Enhanced Bayesian Upper Confidence Bound Framework](https://mlsanity.com/paper/2201.12955/) (2022)
- [Differentiable Annealed Importance Sampling Minimizes The Symmetrized Kullback-Leibler Divergence Between Initial and Target Distribution](https://mlsanity.com/paper/2405.14840/) (2024)
- [Fixed-Mean Gaussian Processes for Post-hoc Bayesian Deep Learning](https://mlsanity.com/paper/2412.04177/) (2024)
- [Amortized Simulation-Based Inference in Generalized Bayes via Neural Posterior Estimation](https://mlsanity.com/paper/2601.22367/) (2026)
- [Correcting Model Bias with Sparse Implicit Processes](https://mlsanity.com/paper/2207.10673/) (2022)
- [Variational Bayesian Neural Networks via Resolution of Singularities](https://mlsanity.com/paper/2302.06035/) (2023)
- [Parameter Estimation in DAGs from Incomplete Data via Optimal Transport](https://mlsanity.com/paper/2305.15927/) (2023)
- [Alpha-divergence loss function for neural density ratio estimation](https://mlsanity.com/paper/2402.02041/) (2024)
- [Alpha Entropy Search for New Information-based Bayesian Optimization](https://mlsanity.com/paper/2411.16586/) (2024)
- [Symmetry Guarantees Statistic Recovery in Variational Inference](https://mlsanity.com/paper/2604.18310/) (2026)
- [Flow-Transformed Implicit Processes for Function-Space Variational Inference](https://mlsanity.com/paper/2606.01954/) (2026)
- [Nonlinear Kalman Filtering with Reparametrization Gradients](https://mlsanity.com/paper/2303.04450/) (2023)
- [Learning variational autoencoders via MCMC speed measures](https://mlsanity.com/paper/2308.13731/) (2023)
- [Stability-based Generalization Bounds for Variational Inference](https://mlsanity.com/paper/2502.12353/) (2025)

## Abstract (arXiv)

Black-box alpha (BB-α) is a new approximate inference method based on the minimization of α-divergences. BB-α scales to large datasets because it can be implemented using stochastic gradient descent. BB-α can be applied to complex probabilistic models with little effort since it only requires as input the likelihood function and its gradients. These gradients can be easily obtained using automatic differentiation. By changing the divergence parameter α, the method is able to interpolate between variational Bayes (VB) (α rightarrow 0) and an algorithm similar to expectation propagation (EP) (α = 1). Experiments on probit regression and neural network regression and classification problems show that BB-α with non-standard settings of α, such as α = 0.5, usually produces better predictions than with α rightarrow 0 (VB) or α = 1 (EP).