mlsanityData to 14 Oct 2026

On the geometry of Stein variational gradient descent

arXiv 1912.00894 ↗First posted 2 Dec 2019stat.ML cs.LG math.APcited by 82

Not on the map. Why?

Abstract (arXiv)

Bayesian inference problems require sampling or approximating high-dimensional probability distributions. The focus of this paper is on the recently introduced Stein variational gradient descent methodology, a class of algorithms that rely on iterated steepest descent steps with respect to a reproducing kernel Hilbert space norm. This construction leads to interacting particle systems, the mean-field limit of which is a gradient flow on the space of probability distributions equipped with a certain geometrical structure. We leverage this viewpoint to shed some light on the convergence properties of the algorithm, in particular addressing the problem of choosing a suitable positive definite kernel function. Our analysis leads us to considering certain nondifferentiable kernels with adjusted tails. We demonstrate significant performance gains of these in various numerical experiments.

Key facts. Cited by 82 papers in the map (data through 14 Oct 2026). First posted on arXiv 2 Dec 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 17; 2022: 14; 2023: 15; 2024: 14; 2025: 10; 2026: 12.

82 papers.

Sort:

By area:

  1. 2021Repulsive Deep Ensembles are Bayesiancited by 104
  2. 2022Variational inference via Wasserstein gradient flowscited by 85
  3. 2021Kernel Stein Discrepancy Descentcited by 57
  4. 2021Optimizing Functionals on the Space of Probabilities with Input Convex Neural Networkscited by 53
  5. 2023Forward-backward Gaussian variational inference via JKO in the Bures-Wasserstein Spacecited by 36
  6. 2022Birth-death dynamics for sampling: Global convergence, approximations and their asymptoticscited by 30
  7. 2023A mean-field games laboratory for generative modelingcited by 28
  8. 2024Sampling in Unit Time with Kernel Fisher-Rao Flowcited by 27
  9. 2021Annealed Stein Variational Gradient Descentcited by 26
  10. 2022Particle algorithms for maximum likelihood training of latent variable modelscited by 26
  11. 2021Projected Wasserstein gradient descent for high-dimensional Bayesian inferencecited by 24
  12. 2022A Finite-Particle Convergence Rate for Stein Variational Gradient Descentcited by 22
38 more
  1. 2021A Convergence Theory for SVGD in the Population Limit under Talagrand's Inequality T1cited by 21
  2. 2021A Neural Tangent Kernel Perspective of GANscited by 21
  3. 2021On Stein Variational Neural Network Ensemblescited by 21
  4. 2021Efficient Gradient Flows in Sliced-Wasserstein Spacecited by 21
  5. 2023Unifying GANs and Score-Based Diffusion as Generative Particle Modelscited by 21
  6. 2022Particle-based Variational Inference with Preconditioned Functional Gradient Flowcited by 20
  7. 2023Sampling via Gradient Flows in the Space of Probability Measurescited by 20
  8. 2022Regularized Stein Variational Gradient Flowcited by 19
  9. 2021Sampling with Mirrored Stein Operatorscited by 17
  10. 2023Gaussian Interpolation Flowscited by 17
  11. 2022Convergence of Stein Variational Gradient Descent under a Weaker Smoothness Conditioncited by 16
  12. 2023Gradient Flows for Sampling: Mean-Field Models, Gaussian Approximations and Affine Invariancecited by 16
  13. 2021Stein Variational Gradient Descent: many-particle and long-time asymptoticscited by 15
  14. 2024Repulsive Latent Score Distillation for Solving Inverse Problemscited by 15
  15. 2024Improved Finite-Particle Convergence Rates for Stein Variational Gradient Descentcited by 14
  16. 2023Towards Understanding the Dynamics of Gaussian-Stein Variational Gradient Descentcited by 13
  17. 2024Error bounds for particle gradient descent, and extensions of the log-Sobolev and Talagrand inequalitiescited by 13
  18. 2023Momentum Particle Maximum Likelihoodcited by 12
  19. 2022Sampling in Constrained Domains with Orthogonal-Space Variational Gradient Descentcited by 11
  20. 2023Tuning-Free Maximum Likelihood Training of Latent Variable Models via Coin Bettingcited by 11
  21. 2024Theoretical Guarantees for Variational Inference with Fixed-Variance Mixture of Gaussianscited by 11
  22. 2024Stein transport for Bayesian inferencecited by 11
  23. 2021Learning Equivariant Energy Based Models with Equivariant Stein Variational Gradient Descentcited by 10
  24. 2023Accelerating optimization over the space of probability measurescited by 10
  25. 2024Interaction-Force Transport Gradient Flowscited by 9
  26. 2021Ensemble Inference Methods for Models With Noisy and Expensive Likelihoodscited by 8
  27. 2023Coin Sampling: Gradient-Based Bayesian Inference without Learning Ratescited by 8
  28. 2022Utilising the CLT Structure in Stochastic Gradient based Sampling : Improved Analysis and Faster Algorithmscited by 7
  29. 2024Kernel Approximation of Fisher-Rao Gradient Flowscited by 7
  30. 2023Provably Fast Finite Particle Variants of SVGD via Virtual Particle Stochastic Approximationcited by 6
  31. 2025DiScoFormer: Plug-In Density and Score Estimation with Transformerscited by 6
  32. 2026Sobolev Regularized MMD Gradient Flowcited by 6
  33. 2021Relative Entropy Gradient Sampler for Unnormalized Distributionscited by 5
  34. 2026Finite-Particle Rates for Regularized Stein Variational Gradient Descentcited by 5
  35. 2021Multilevel Stein variational gradient descent with applications to Bayesian inverse problemscited by 4
  36. 2022A stochastic Stein Variational Newton methodcited by 4
  37. 2022A Note on the Convergence of Mirrored Stein Variational Gradient Descent under (L0,L1)-Smoothness Conditioncited by 4
  38. 2024Analysis of Kernel Mirror Prox for Measure Optimizationcited by 4

50 most cited of 82 All 82 (JSON)