# Kernel Stein Discrepancy Descent

arXiv: https://arxiv.org/abs/2105.09994 · HTML: https://mlsanity.com/paper/2105.09994/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 68 references that are in the map.
- First posted on arXiv 20 May 2021.

## Builds on (most cited first, 23 of 23)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluation](https://mlsanity.com/paper/1602.03253/) (2016)
- [A Kernel Test of Goodness of Fit](https://mlsanity.com/paper/1602.02964/) (2016)
- [Analysis of Langevin Monte Carlo via convex optimization](https://mlsanity.com/paper/1802.09188/) (2018)
- [Maximum Mean Discrepancy Gradient Flow](https://mlsanity.com/paper/1906.04370/) (2019)
- [Sampling as optimization in the space of measures: The Langevin dynamics as a composite optimization problem](https://mlsanity.com/paper/1802.08089/) (2018)
- [Measuring Sample Quality with Kernels](https://mlsanity.com/paper/1703.01717/) (2017)
- [On the geometry of Stein variational gradient descent](https://mlsanity.com/paper/1912.00894/) (2019)
- [SVGD as a kernelized Wasserstein gradient flow of the chi-squared divergence](https://mlsanity.com/paper/2006.02509/) (2020)
- [A Non-Asymptotic Analysis for Stein Variational Gradient Descent](https://mlsanity.com/paper/2006.09797/) (2020)
- [Stein Points](https://mlsanity.com/paper/1803.10161/) (2018)
- [Blindness of score-based methods to isolated components and mixing proportions](https://mlsanity.com/paper/2008.10087/) (2020)
- [Stein Point Markov Chain Monte Carlo](https://mlsanity.com/paper/1905.03673/) (2019)
- [Optimal Thinning of MCMC Output](https://mlsanity.com/paper/2005.03952/) (2020)
- [A Kernel Stein Test for Comparing Latent Variable Models](https://mlsanity.com/paper/1907.00586/) (2019)
- [Scaling-up Empirical Risk Minimization: Optimization of Incomplete U-statistics](https://mlsanity.com/paper/1501.02629/) (2015)
- [A Stein Goodness-of-fit Test for Directional Distributions](https://mlsanity.com/paper/2002.06843/) (2020)
- [Bayesian posterior approximation via greedy particle optimization](https://mlsanity.com/paper/1805.07912/) (2018)
- [The reproducing Stein kernel approach for post-hoc corrected sampling](https://mlsanity.com/paper/2001.09266/) (2020)
- [Measure Transport with Kernel Stein Discrepancy](https://mlsanity.com/paper/2010.11779/) (2020)
- [Differential Inclusions in Wasserstein Spaces: The Cauchy-Lipschitz Framework](https://mlsanity.com/paper/2007.08906/) (2020)
- [The Wasserstein Proximal Gradient Algorithm](https://mlsanity.com/paper/2002.03035/) (2020)

## Built on by (most cited first, 30 of 57)

- [Optimizing Functionals on the Space of Probabilities with Input Convex Neural Networks](https://mlsanity.com/paper/2106.00774/) (2021)
- [Mirror Descent with Relative Smoothness in Measure Spaces, with application to Sinkhorn and EM](https://mlsanity.com/paper/2206.08873/) (2022)
- [A Rigorous Link between Deep Ensembles and (Variational) Bayesian Methods](https://mlsanity.com/paper/2305.15027/) (2023)
- [Birth-death dynamics for sampling: Global convergence, approximations and their asymptotics](https://mlsanity.com/paper/2211.00450/) (2022)
- [Efficient Gradient Flows in Sliced-Wasserstein Space](https://mlsanity.com/paper/2110.10972/) (2021)
- [KSD Aggregated Goodness-of-fit Test](https://mlsanity.com/paper/2202.00824/) (2022)
- [Convergence of mean-field Langevin dynamics: Time and space discretization, stochastic gradient, and variance reduction](https://mlsanity.com/paper/2306.07221/) (2023)
- [Sampling with Mollified Interaction Energy Descent](https://mlsanity.com/paper/2210.13400/) (2022)
- [Towards Understanding the Dynamics of Gaussian-Stein Variational Gradient Descent](https://mlsanity.com/paper/2305.14076/) (2023)
- [Weighted quantization using MMD: From mean field to mean shift via gradient flows](https://mlsanity.com/paper/2502.10600/) (2025)
- [Theoretical Guarantees for Variational Inference with Fixed-Variance Mixture of Gaussians](https://mlsanity.com/paper/2406.04012/) (2024)
- [Stein transport for Bayesian inference](https://mlsanity.com/paper/2409.01464/) (2024)
- [Fast Bayesian Coresets via Subsampling and Quasi-Newton Refinement](https://mlsanity.com/paper/2203.09675/) (2022)
- [Kernel Stein Discrepancy thinning: a theoretical perspective of pathologies and a practical fix with regularization](https://mlsanity.com/paper/2301.13528/) (2023)
- [GAD-PVI: A General Accelerated Dynamic-Weight Particle-Based Variational Inference Framework](https://mlsanity.com/paper/2312.16429/) (2023)
- [Interaction-Force Transport Gradient Flows](https://mlsanity.com/paper/2405.17075/) (2024)
- [A Dictionary of Closed-Form Kernel Mean Embeddings](https://mlsanity.com/paper/2504.18830/) (2025)
- [Coin Sampling: Gradient-Based Bayesian Inference without Learning Rates](https://mlsanity.com/paper/2301.11294/) (2023)
- [Kernel Semi-Implicit Variational Inference](https://mlsanity.com/paper/2405.18997/) (2024)
- [KerJEPA: Kernel Discrepancies for Euclidean Self-Supervised Learning](https://mlsanity.com/paper/2512.19605/) (2025)
- [DPVI: A Dynamic-Weight Particle-Based Variational Inference Framework](https://mlsanity.com/paper/2112.00945/) (2021)
- [Particle Semi-Implicit Variational Inference](https://mlsanity.com/paper/2407.00649/) (2024)
- [Kernel Approximation of Fisher-Rao Gradient Flows](https://mlsanity.com/paper/2410.20622/) (2024)
- [Sobolev Regularized MMD Gradient Flow](https://mlsanity.com/paper/2605.11884/) (2026)
- [Efficient displacement convex optimization with particle gradient descent](https://mlsanity.com/paper/2302.04753/) (2023)
- [A Practical Diffusion Path for Sampling](https://mlsanity.com/paper/2406.14040/) (2024)
- [Analysis of Kernel Mirror Prox for Measure Optimization](https://mlsanity.com/paper/2403.00147/) (2024)
- [Inclusive KL Gradient Flows: Otto-Wasserstein, Fisher-Rao-Gaussian, and Local-Estimator Dynamics](https://mlsanity.com/paper/2411.00214/) (2024)
- [A note on the unique properties of the Kullback--Leibler divergence for sampling via gradient flows](https://mlsanity.com/paper/2507.04330/) (2025)
- [A Deterministic Sampling Method via Maximum Mean Discrepancy Flow with Adaptive Kernel](https://mlsanity.com/paper/2111.10722/) (2021)

## Abstract (arXiv)

Among dissimilarities between probability distributions, the Kernel Stein Discrepancy (KSD) has received much interest recently. We investigate the properties of its Wasserstein gradient flow to approximate a target probability distribution π on ℝd, known up to a normalization constant. This leads to a straightforwardly implementable, deterministic score-based method to sample from π, named KSD Descent, which uses a set of particles to approximate π. Remarkably, owing to a tractable loss function, KSD Descent can leverage robust parameter-free optimization schemes such as L-BFGS; this contrasts with other popular particle-based schemes such as the Stein Variational Gradient Descent algorithm. We study the convergence properties of KSD Descent and demonstrate its practical relevance. However, we also highlight failure cases by showing that the algorithm can get stuck in spurious local minima.