# Adaptive importance sampling for heavy-tailed distributions via α-divergence minimization

arXiv: https://arxiv.org/abs/2310.16653 · HTML: https://mlsanity.com/paper/2310.16653/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 90 references that are in the map.
- First posted on arXiv 25 Oct 2023.

## Builds on (most cited first, 24 of 24)

- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Importance Weighted Autoencoders](https://mlsanity.com/paper/1509.00519/) (2015)
- [Maximizing acquisition functions for Bayesian optimization](https://mlsanity.com/paper/1805.10196/) (2018)
- [R'enyi Divergence Variational Inference](https://mlsanity.com/paper/1602.02311/) (2016)
- [Variational inference for Monte Carlo objectives](https://mlsanity.com/paper/1602.06725/) (2016)
- [Importance Weighting and Variational Inference](https://mlsanity.com/paper/1808.09034/) (2018)
- [Black-box α-divergence Minimization](https://mlsanity.com/paper/1511.03243/) (2015)
- [Challenges and Opportunities in High-dimensional Variational Inference](https://mlsanity.com/paper/2103.01085/) (2021)
- [Marginal likelihood computation for model selection and hypothesis testing: an extensive review](https://mlsanity.com/paper/2005.08334/) (2020)
- [Pathfinder: Parallel quasi-Newton variational inference](https://mlsanity.com/paper/2108.03782/) (2021)
- [Variational Inference with Tail-adaptive f-Divergence](https://mlsanity.com/paper/1810.11943/) (2018)
- [Emulation of physical processes with Emukit](https://mlsanity.com/paper/2110.13293/) (2021)
- [Variational Representations and Neural Network Estimation of R'enyi Divergences](https://mlsanity.com/paper/2007.03814/) (2020)
- [Layered Adaptive Importance Sampling](https://mlsanity.com/paper/1505.04732/) (2015)
- [Adaptive Importance Sampling meets Mirror Descent: a Bias-variance tradeoff](https://mlsanity.com/paper/2110.15590/) (2021)
- [Convergence rates for optimised adaptive importance samplers](https://mlsanity.com/paper/1903.12044/) (2019)
- [Multiple Importance Sampling ELBO and Deep Ensembles of Variational Approximations](https://mlsanity.com/paper/2202.10951/) (2022)
- [MixFlows: principled variational inference via mixed flows](https://mlsanity.com/paper/2205.07475/) (2022)
- [Asymptotic optimality of adaptive importance sampling](https://mlsanity.com/paper/1806.00989/) (2018)
- [A Variational Bayesian Approach for Image Restoration. Application to Image Deblurring with Poisson-Gaussian Noise](https://mlsanity.com/paper/1610.07519/) (2016)
- [Monotonic Alpha-divergence Minimisation for Variational Inference](https://mlsanity.com/paper/2103.05684/) (2021)
- [Uphill Roads to Variational Tightness: Monotonicity and Monte Carlo Objectives](https://mlsanity.com/paper/2201.10989/) (2022)
- [Importance Sampled Stochastic Optimization for Variational Inference](https://mlsanity.com/paper/1704.05786/) (2017)
- [A principled stopping rule for importance sampling](https://mlsanity.com/paper/2108.13289/) (2021)

## Built on by (most cited first, 3 of 3)

- [How to Approximate Inference with Subtractive Mixture Models](https://mlsanity.com/paper/2604.16714/) (2026)
- [Even More Guarantees for Variational Inference in the Presence of Symmetries](https://mlsanity.com/paper/2604.21407/) (2026)
- [Quantifying Behavioral Tails in Black-Box Language Models](https://mlsanity.com/paper/2609.33638/) (2026)

## Abstract (arXiv)

Adaptive importance sampling (AIS) algorithms are widely used to approximate expectations with respect to complicated target probability distributions. When the target has heavy tails, existing AIS algorithms can provide inconsistent estimators or exhibit slow convergence, as they often neglect the target's tail behaviour. To avoid this pitfall, we propose an AIS algorithm that approximates the target by Student-t proposal distributions. We adapt location and scale parameters by matching the escort moments - which are defined even for heavy-tailed distributions - of the target and the proposal. These updates minimize the α-divergence between the target and the proposal, thereby connecting with variational inference. We then show that the α-divergence can be approximated by a generalized notion of effective sample size and leverage this new perspective to adapt the tail parameter with Bayesian optimization. We demonstrate the efficacy of our approach through applications to synthetic targets and a Bayesian Student-t regression task on a real example with clinical trial data.