mlsanityData to 14 Oct 2026

Kernel Stein Discrepancy Descent

arXiv 2105.09994 ↗First posted 20 May 2021stat.ML cs.LGcited by 57

Not on the map. Why?

Abstract (arXiv)

Among dissimilarities between probability distributions, the Kernel Stein Discrepancy (KSD) has received much interest recently. We investigate the properties of its Wasserstein gradient flow to approximate a target probability distribution π on ℝd, known up to a normalization constant. This leads to a straightforwardly implementable, deterministic score-based method to sample from π, named KSD Descent, which uses a set of particles to approximate π. Remarkably, owing to a tractable loss function, KSD Descent can leverage robust parameter-free optimization schemes such as L-BFGS; this contrasts with other popular particle-based schemes such as the Stein Variational Gradient Descent algorithm. We study the convergence properties of KSD Descent and demonstrate its practical relevance. However, we also highlight failure cases by showing that the algorithm can get stuck in spurious local minima.

Key facts. Cited by 57 papers in the map (data through 14 Oct 2026). Builds on 23 of its 68 references that are in the map. First posted on arXiv 20 May 2021.

Builds on

23 of 68 references.

References · 23

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 507
  3. 2016A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluationcited by 236
  4. 2016A Kernel Test of Goodness of Fitcited by 143
  5. 2018Analysis of Langevin Monte Carlo via convex optimizationcited by 127
  6. 2019Maximum Mean Discrepancy Gradient Flowcited by 127
  7. 2018Sampling as optimization in the space of measures: The Langevin dynamics as a composite optimization problemcited by 123
  8. 2017Measuring Sample Quality with Kernelscited by 119
  9. 2019On the geometry of Stein variational gradient descentcited by 82
  10. 2020SVGD as a kernelized Wasserstein gradient flow of the chi-squared divergencecited by 64
  11. 2020A Non-Asymptotic Analysis for Stein Variational Gradient Descentcited by 61
  12. 2018Stein Pointscited by 43
11 more
  1. 2020Blindness of score-based methods to isolated components and mixing proportionscited by 34
  2. 2019Stein Point Markov Chain Monte Carlocited by 29
  3. 2020Optimal Thinning of MCMC Outputcited by 26
  4. 2019A Kernel Stein Test for Comparing Latent Variable Modelscited by 18
  5. 2015Scaling-up Empirical Risk Minimization: Optimization of Incomplete U-statisticscited by 17
  6. 2020A Stein Goodness-of-fit Test for Directional Distributionscited by 13
  7. 2018Bayesian posterior approximation via greedy particle optimizationcited by 10
  8. 2020The reproducing Stein kernel approach for post-hoc corrected samplingcited by 10
  9. 2020Measure Transport with Kernel Stein Discrepancycited by 8
  10. 2020Differential Inclusions in Wasserstein Spaces: The Cauchy-Lipschitz Frameworkcited by 4
  11. 2020The Wasserstein Proximal Gradient Algorithmcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 7; 2023: 9; 2024: 14; 2025: 14; 2026: 9.

57 papers.

Sort:

By area:

  1. 2021Optimizing Functionals on the Space of Probabilities with Input Convex Neural Networkscited by 53
  2. 2022Mirror Descent with Relative Smoothness in Measure Spaces, with application to Sinkhorn and EMcited by 34
  3. 2023A Rigorous Link between Deep Ensembles and (Variational) Bayesian Methodscited by 32
  4. 2022Birth-death dynamics for sampling: Global convergence, approximations and their asymptoticscited by 30
  5. 2021Efficient Gradient Flows in Sliced-Wasserstein Spacecited by 21
  6. 2022KSD Aggregated Goodness-of-fit Testcited by 21
  7. 2023Convergence of mean-field Langevin dynamics: Time and space discretization, stochastic gradient, and variance reductioncited by 20
  8. 2022Sampling with Mollified Interaction Energy Descentcited by 17
  9. 2023Towards Understanding the Dynamics of Gaussian-Stein Variational Gradient Descentcited by 13
  10. 2025Weighted quantization using MMD: From mean field to mean shift via gradient flowscited by 12
  11. 2024Theoretical Guarantees for Variational Inference with Fixed-Variance Mixture of Gaussianscited by 11
  12. 2024Stein transport for Bayesian inferencecited by 11
38 more
  1. 2022Fast Bayesian Coresets via Subsampling and Quasi-Newton Refinementcited by 9
  2. 2023Kernel Stein Discrepancy thinning: a theoretical perspective of pathologies and a practical fix with regularizationcited by 9
  3. 2023GAD-PVI: A General Accelerated Dynamic-Weight Particle-Based Variational Inference Frameworkcited by 9
  4. 2024Interaction-Force Transport Gradient Flowscited by 9
  5. 2025A Dictionary of Closed-Form Kernel Mean Embeddingscited by 9
  6. 2023Coin Sampling: Gradient-Based Bayesian Inference without Learning Ratescited by 8
  7. 2024Kernel Semi-Implicit Variational Inferencecited by 8
  8. 2025KerJEPA: Kernel Discrepancies for Euclidean Self-Supervised Learningcited by 8
  9. 2021DPVI: A Dynamic-Weight Particle-Based Variational Inference Frameworkcited by 7
  10. 2024Particle Semi-Implicit Variational Inferencecited by 7
  11. 2024Kernel Approximation of Fisher-Rao Gradient Flowscited by 7
  12. 2026Sobolev Regularized MMD Gradient Flowcited by 6
  13. 2023Efficient displacement convex optimization with particle gradient descentcited by 5
  14. 2024A Practical Diffusion Path for Samplingcited by 5
  15. 2024Analysis of Kernel Mirror Prox for Measure Optimizationcited by 4
  16. 2024Inclusive KL Gradient Flows: Otto-Wasserstein, Fisher-Rao-Gaussian, and Local-Estimator Dynamicscited by 4
  17. 2025A note on the unique properties of the Kullback--Leibler divergence for sampling via gradient flowscited by 4
  18. 2021A Deterministic Sampling Method via Maximum Mean Discrepancy Flow with Adaptive Kernelcited by 3
  19. 2022Benchmarking Bayesian neural networks and evaluation metrics for regression taskscited by 3
  20. 2023Learning Rate Free Sampling in Constrained Domainscited by 3
  21. 2024Data-centric Prediction Explanation via Kernelized Stein Discrepancycited by 3
  22. 2024Statistical and Geometrical properties of regularized Kernel Kullback-Leibler divergencecited by 3
  23. 2025Towards Understanding Gradient Dynamics of the Sliced-Wasserstein Distance via Critical Point Analysiscited by 3
  24. 2025The Minimax Lower Bound of Kernel Stein Discrepancy Estimationcited by 3
  25. 2022On the failure of variational score matching for VAE modelscited by 2
  26. 2024Stein Boltzmann Sampling: A Variational Approach for Global Optimizationcited by 2
  27. 2025Smoothed Distance Kernels for MMDs and Applications in Wasserstein Gradient Flowscited by 2
  28. 2025Stationary MMD Pointscited by 2
  29. 2025Robust inference using density-powered Stein operatorscited by 2
  30. 2026To discretize continually: Mean shift interacting particle systems for Bayesian inferencecited by 2
  31. 2026Minimax Lower Bounds of Kernel Discrepancy Estimation: MMD, HSIC, KSDcited by 2
  32. 2023Bayes Hilbert Spaces for Posterior Approximationcited by 1
  33. 2024Monte Carlo with kernel-based Gibbs measures: Guarantees for probabilistic herdingcited by 1
  34. 2024Semi-Implicit Functional Gradient Flow for Efficient Samplingcited by 1
  35. 2025Optimizing Kernel Discrepancies via Subset Selectioncited by 1
  36. 2026A Kernel Approach for Semi-implicit Variational Inferencecited by 1
  37. 2026Thinned Mean Field Langevin Dynamicscited by 1
  38. 2026A Gradient Flow Perspective on Minimum MMD Estimationcited by 1

50 most cited of 57 All 57 (JSON)