# Pareto Smoothed Importance Sampling

arXiv: https://arxiv.org/abs/1507.02646 · HTML: https://mlsanity.com/paper/1507.02646/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jul 2015.

## Built on by (most cited first, 30 of 42)

- [Variational Refinement for Importance Sampling Using the Forward Kullback-Leibler Divergence](https://mlsanity.com/paper/2106.15980/) (2021)
- [Pathfinder: Parallel quasi-Newton variational inference](https://mlsanity.com/paper/2108.03782/) (2021)
- [Sensitivity-Aware Amortized Bayesian Inference](https://mlsanity.com/paper/2310.11122/) (2023)
- [Bayesian hierarchical stacking: Some models are (somewhere) useful](https://mlsanity.com/paper/2101.08954/) (2021)
- [Amortized Bayesian Multilevel Models](https://mlsanity.com/paper/2408.13230/) (2024)
- [q-Paths: Generalizing the Geometric Annealing Path using Power Means](https://mlsanity.com/paper/2107.00745/) (2021)
- [Variational Inference for Uncertainty Quantification: an Analysis of Trade-offs](https://mlsanity.com/paper/2403.13748/) (2024)
- [Robust Amortized Bayesian Inference with Self-Consistency Losses on Unlabeled Data](https://mlsanity.com/paper/2501.13483/) (2025)
- [Variational Inference in Location-Scale Families: Exact Recovery of the Mean and Correlation Matrix](https://mlsanity.com/paper/2410.11067/) (2024)
- [BayesFlow 2: Multi-Backend Amortized Bayesian Inference in Python](https://mlsanity.com/paper/2602.07098/) (2026)
- [The computational asymptotics of Gaussian variational inference and the Laplace approximation](https://mlsanity.com/paper/2104.05886/) (2021)
- [Mitigating mode collapse in normalizing flows by annealing with an adaptive schedule: Application to parameter estimation](https://mlsanity.com/paper/2505.03652/) (2025)
- [Reinforced sequential Monte Carlo for amortised sampling](https://mlsanity.com/paper/2510.11711/) (2025)
- [Mathematical Theory of Bayesian Statistics for Unknown Information Source](https://mlsanity.com/paper/2206.05630/) (2022)
- [Meta-Uncertainty in Bayesian Model Comparison](https://mlsanity.com/paper/2210.07278/) (2022)
- [Mitigating Statistical Bias within Differentially Private Synthetic Data](https://mlsanity.com/paper/2108.10934/) (2021)
- [Demonstrating the power and flexibility of variational assumptions for amortized neural posterior estimation in environmental applications](https://mlsanity.com/paper/2404.10899/) (2024)
- [Uphill Roads to Variational Tightness: Monotonicity and Monte Carlo Objectives](https://mlsanity.com/paper/2201.10989/) (2022)
- [Fast post-process Bayesian inference with Variational Sparse Bayesian Quadrature](https://mlsanity.com/paper/2303.05263/) (2023)
- [Amortized Bayesian Workflow](https://mlsanity.com/paper/2409.04332/) (2024)
- [Amortized Bayesian Mixture Models](https://mlsanity.com/paper/2501.10229/) (2025)
- [PriorGuide: Test-Time Prior Adaptation for Simulation-Based Inference](https://mlsanity.com/paper/2510.13763/) (2025)
- [Generalized Guarantees for Variational Inference in the Presence of Even and Elliptical Symmetry](https://mlsanity.com/paper/2511.01064/) (2025)
- [Efficient Bayes Inference in Neural Networks through Adaptive Importance Sampling](https://mlsanity.com/paper/2210.00993/) (2022)
- [Bayesian score calibration for approximate models](https://mlsanity.com/paper/2211.05357/) (2022)
- [A Heavy-Tailed Algebra for Probabilistic Programming](https://mlsanity.com/paper/2306.09262/) (2023)
- [Differentiable Pareto-Smoothed Weighting for High-Dimensional Heterogeneous Treatment Effect Estimation](https://mlsanity.com/paper/2404.17483/) (2024)
- [FlowVAT: Normalizing Flow Variational Inference with Affine-Invariant Tempering](https://mlsanity.com/paper/2505.10466/) (2025)
- [Normalizing Flow Regression for Bayesian Inference with Offline Likelihood Evaluations](https://mlsanity.com/paper/2504.11554/) (2025)
- [Temper-Then-Tilt: Principled Unlearning for Generative Models through Tempering and Classifier Guidance](https://mlsanity.com/paper/2602.10217/) (2026)

## Abstract (arXiv)

Importance weighting is a general way to adjust Monte Carlo integration to account for draws from the wrong distribution, but the resulting estimate can be highly variable when the importance ratios have a heavy right tail. This routinely occurs when there are aspects of the target distribution that are not well captured by the approximating distribution, in which case more stable estimates can be obtained by modifying extreme importance ratios. We present a new method for stabilizing importance weights using a generalized Pareto distribution fit to the upper tail of the distribution of the simulated importance ratios. The method, which empirically performs better than existing methods for stabilizing importance sampling estimates, includes stabilized effective sample size estimates, Monte Carlo error estimates, and convergence diagnostics. The presented Pareto hatk finite sample convergence rate diagnostic is useful for any Monte Carlo estimator.