mlsanityData to 7 Oct 2026

Stein Variational Gradient Descent: many-particle and long-time asymptotics

arXiv 2102.12956 ↗First posted 25 Feb 2021stat.ML cs.LG cs.NAcited by 15

Not on the map. Why?

Abstract (arXiv)

Stein variational gradient descent (SVGD) refers to a class of methods for Bayesian inference based on interacting particle systems. In this paper, we consider the originally proposed deterministic dynamics as well as a stochastic variant, each of which represent one of the two main paradigms in Bayesian computational statistics: variational inference and Markov chain Monte Carlo. As it turns out, these are tightly linked through a correspondence between gradient flow structures and large-deviation principles rooted in statistical physics. To expose this relationship, we develop the cotangent space construction for the Stein geometry, prove its basic properties, and determine the large-deviation functional governing the many-particle limit for the empirical measure. Moreover, we identify the Stein-Fisher information (or kernelised Stein discrepancy) as its leading order contribution in the long-time and many-particle regime in the sense of Gamma-convergence, shedding some light on the finite-particle properties of SVGD. Finally, we establish a comparison principle between the Stein-Fisher information and RKHS-norms that might be of independent interest.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 24 of its 90 references that are in the map. First posted on arXiv 25 Feb 2021.

Builds on

24 of 90 references.

References · 24

  1. 2016Variational Inference: A Review for Statisticianscited by 1,466
  2. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 498
  3. 2016A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluationcited by 236
  4. 2017Advances in Variational Inferencecited by 236
  5. 2017Stein Variational Gradient Descent as Gradient Flowcited by 149
  6. 2015A Complete Recipe for Stochastic Gradient MCMCcited by 146
  7. 2016A Kernel Test of Goodness of Fitcited by 143
  8. 2019Maximum Mean Discrepancy Gradient Flowcited by 123
  9. 2017Measuring Sample Quality with Kernelscited by 119
  10. 2019Theoretical guarantees for sampling and inference in generative models with latent diffusionscited by 97
  11. 2017Variational Sequential Monte Carlocited by 93
  12. 2017Filtering Variational Objectivescited by 91
12 more
  1. 2019On the geometry of Stein variational gradient descentcited by 82
  2. 2020Solving high-dimensional Hamilton-Jacobi-Bellman PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path spacecited by 80
  3. 2018Sliced-Wasserstein Flows: Nonparametric Generative Modeling via Optimal Transport and Diffusionscited by 77
  4. 2020SVGD as a kernelized Wasserstein gradient flow of the chi-squared divergencecited by 64
  5. 2020A Non-Asymptotic Analysis for Stein Variational Gradient Descentcited by 61
  6. 2018A Unified Particle-Optimization Framework for Scalable Bayesian Samplingcited by 44
  7. 2018Scaling limit of the Stein variational gradient descent: the mean field regimecited by 35
  8. 2019A Contrastive Divergence for Combining Variational Inference and MCMCcited by 20
  9. 2018Stochastic Gradient MCMC with Repulsive Forcescited by 18
  10. 2018Stein Variational Gradient Descent as Moment Matchingcited by 15
  11. 2020Measure Transport with Kernel Stein Discrepancycited by 8
  12. 2018Wasserstein variational gradient descent: From semi-discrete optimal transport to ensemble variational inferencecited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 1; 2023: 2; 2024: 2; 2025: 4; 2026: 5.

15 papers.

Sort:

  1. 2021A Convergence Theory for SVGD in the Population Limit under Talagrand's Inequality T1cited by 21
  2. 2022Convergence of Stein Variational Gradient Descent under a Weaker Smoothness Conditioncited by 16
  3. 2024Stein transport for Bayesian inferencecited by 11
  4. 2023GAD-PVI: A General Accelerated Dynamic-Weight Particle-Based Variational Inference Frameworkcited by 9
  5. 2023Provably Fast Finite Particle Variants of SVGD via Virtual Particle Stochastic Approximationcited by 6
  6. 2025Accelerated Stein Variational Gradient Flowcited by 5
  7. 2024Long-time asymptotics of noisy SVGD outside the population limitcited by 2
  8. 2025Smoothed Distance Kernels for MMDs and Applications in Wasserstein Gradient Flowscited by 2
  9. 2025Towards understanding Accelerated Stein Variational Gradient Flow -- Analysis of Generalized Bilinear Kernels for Gaussian target distributionscited by 2
  10. 2026Radon--Wasserstein Gradient Flows for Interacting-Particle Sampling in High Dimensionscited by 1
  11. 2026Quantitative Local Convergence of Mean-Field Stein Variational Gradient Flowcited by 1
  12. 2025Adaptive Kernel Selection for Stein Variational Gradient Descentcited by 0
3 more
  1. 2026Well-Posed KL-Regularized Control via Wasserstein and Kalman-Wasserstein KL Divergencescited by 0
  2. 2026Riesz-Kernel Stein Variational Gradient Descent: Renormalized Entropy and Long-Time Particle Limitscited by 0
  3. 2026Interacting particle guidance for sampling reward-tilted generative priorscited by 0