# Kernel Stein Discrepancy thinning: a theoretical perspective of pathologies and a practical fix with regularization

arXiv: https://arxiv.org/abs/2301.13528 · HTML: https://mlsanity.com/paper/2301.13528/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 32 references that are in the map.
- First posted on arXiv 31 Jan 2023.

## Builds on (most cited first, 15 of 15)

- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluation](https://mlsanity.com/paper/1602.03253/) (2016)
- [A Kernel Test of Goodness of Fit](https://mlsanity.com/paper/1602.02964/) (2016)
- [Measuring Sample Quality with Kernels](https://mlsanity.com/paper/1703.01717/) (2017)
- [Measuring Sample Quality with Stein's Method](https://mlsanity.com/paper/1506.03039/) (2015)
- [Kernel Stein Discrepancy Descent](https://mlsanity.com/paper/2105.09994/) (2021)
- [Stein Points](https://mlsanity.com/paper/1803.10161/) (2018)
- [Blindness of score-based methods to isolated components and mixing proportions](https://mlsanity.com/paper/2008.10087/) (2020)
- [Stein Point Markov Chain Monte Carlo](https://mlsanity.com/paper/1905.03673/) (2019)
- [Optimal Thinning of MCMC Output](https://mlsanity.com/paper/2005.03952/) (2020)
- [A Framework for Adaptive MCMC Targeting Multimodal Distributions](https://mlsanity.com/paper/1812.02609/) (2018)
- [Scalable Nonparametric Sampling from Multimodal Posteriors with the Posterior Bootstrap](https://mlsanity.com/paper/1902.03175/) (2019)
- [Post-Processing of MCMC](https://mlsanity.com/paper/2103.16048/) (2021)
- [Posterior Integration on a Riemannian Manifold](https://mlsanity.com/paper/1712.01793/) (2017)
- [Fast compression of MCMC output](https://mlsanity.com/paper/2107.04552/) (2021)

## Built on by (most cited first, 9 of 9)

- [Kernel Semi-Implicit Variational Inference](https://mlsanity.com/paper/2405.18997/) (2024)
- [A Practical Diffusion Path for Sampling](https://mlsanity.com/paper/2406.14040/) (2024)
- [Monte Carlo with kernel-based Gibbs measures: Guarantees for probabilistic herding](https://mlsanity.com/paper/2402.11736/) (2024)
- [Optimizing Kernel Discrepancies via Subset Selection](https://mlsanity.com/paper/2511.02706/) (2025)
- [A Kernel Approach for Semi-implicit Variational Inference](https://mlsanity.com/paper/2601.12023/) (2026)
- [Correcting Mode Proportion Bias in Generalized Bayesian Inference via a Weighted Kernel Stein Discrepancy](https://mlsanity.com/paper/2503.02108/) (2025)
- [Branching Stein Variational Gradient Descent for sampling multimodal distributions](https://mlsanity.com/paper/2506.13916/) (2025)
- [Quenched large deviations for Monte Carlo integration with Coulomb gases](https://mlsanity.com/paper/2508.01392/) (2025)
- [StAD: Stein Amortized Divergence for Fast Likelihoods with Diffusion and Flow](https://mlsanity.com/paper/2605.16486/) (2026)

## Abstract (arXiv)

Stein thinning is a promising algorithm proposed by (Riabiz et al., 2022) for post-processing outputs of Markov chain Monte Carlo (MCMC). The main principle is to greedily minimize the kernelized Stein discrepancy (KSD), which only requires the gradient of the log-target distribution, and is thus well-suited for Bayesian inference. The main advantages of Stein thinning are the automatic remove of the burn-in period, the correction of the bias introduced by recent MCMC algorithms, and the asymptotic properties of convergence towards the target distribution. Nevertheless, Stein thinning suffers from several empirical pathologies, which may result in poor approximations, as observed in the literature. In this article, we conduct a theoretical analysis of these pathologies, to clearly identify the mechanisms at stake, and suggest improved strategies. Then, we introduce the regularized Stein thinning algorithm to alleviate the identified pathologies. Finally, theoretical guarantees and extensive experiments show the high efficiency of the proposed algorithm. An implementation of regularized Stein thinning as the kernax library in python and JAX is available at https://gitlab.com/drti/kernax.