mlsanityData to 7 Oct 2026

A Non-Asymptotic Analysis for Stein Variational Gradient Descent

arXiv 2006.09797 ↗First posted 17 Jun 2020stat.ML cs.LGcited by 61

Not on the map. Why?

Abstract (arXiv)

We study the Stein Variational Gradient Descent (SVGD) algorithm, which optimises a set of particles to approximate a target probability distribution πpropto e-V on ℝd. In the population limit, SVGD performs gradient descent in the space of probability distributions on the KL divergence with respect to π, where the gradient is smoothed through a kernel integral operator. In this paper, we provide a novel finite time analysis for the SVGD algorithm. We provide a descent lemma establishing that the algorithm decreases the objective at each iteration, and rates of convergence for the average Stein Fisher divergence (also referred to as Kernel Stein Discrepancy). We also provide a convergence result of the finite particle system corresponding to the practical implementation of SVGD to its population version.

Key facts. Cited by 61 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 13; 2023: 13; 2024: 9; 2025: 5; 2026: 8.

61 papers.

Sort:

By area:

  1. 2021Repulsive Deep Ensembles are Bayesiancited by 103
  2. 2021Priors in Bayesian Deep Learning: A Reviewcited by 101
  3. 2021Kernel Stein Discrepancy Descentcited by 57
  4. 2022Sequential Neural Score Estimation: Likelihood-Free Inference with Conditional Score Based Diffusion Modelscited by 50
  5. 2023Forward-backward Gaussian variational inference via JKO in the Bures-Wasserstein Spacecited by 36
  6. 2022Mirror Descent with Relative Smoothness in Measure Spaces, with application to Sinkhorn and EMcited by 33
  7. 2022A Finite-Particle Convergence Rate for Stein Variational Gradient Descentcited by 22
  8. 2021A Convergence Theory for SVGD in the Population Limit under Talagrand's Inequality T1cited by 21
  9. 2021On Stein Variational Neural Network Ensemblescited by 21
  10. 2022Regularized Stein Variational Gradient Flowcited by 18
  11. 2021Sampling with Mirrored Stein Operatorscited by 17
  12. 2022Sampling with Mollified Interaction Energy Descentcited by 17
38 more
  1. 2022Convergence of Stein Variational Gradient Descent under a Weaker Smoothness Conditioncited by 16
  2. 2023Gradient Flows for Sampling: Mean-Field Models, Gaussian Approximations and Affine Invariancecited by 16
  3. 2021Stein Variational Gradient Descent: many-particle and long-time asymptoticscited by 15
  4. 2023Towards Understanding the Dynamics of Gaussian-Stein Variational Gradient Descentcited by 13
  5. 2024Improved Finite-Particle Convergence Rates for Stein Variational Gradient Descentcited by 13
  6. 2023Tuning-Free Maximum Likelihood Training of Latent Variable Models via Coin Bettingcited by 11
  7. 2024Theoretical Guarantees for Variational Inference with Fixed-Variance Mixture of Gaussianscited by 11
  8. 2024Stein transport for Bayesian inferencecited by 11
  9. 2023Particle-based Variational Inference with Generalized Wasserstein Gradient Flowcited by 10
  10. 2022Batch Bayesian optimisation via density-ratio estimation with guaranteescited by 9
  11. 2023GAD-PVI: A General Accelerated Dynamic-Weight Particle-Based Variational Inference Frameworkcited by 9
  12. 2023Coin Sampling: Gradient-Based Bayesian Inference without Learning Ratescited by 8
  13. 2021DPVI: A Dynamic-Weight Particle-Based Variational Inference Frameworkcited by 7
  14. 2023Geometry of Score Based Generative Modelscited by 7
  15. 2023Particle-based Online Bayesian Samplingcited by 7
  16. 2022A blob method for inhomogeneous diffusion with applications to multi-agent control and samplingcited by 6
  17. 2023Provably Fast Finite Particle Variants of SVGD via Virtual Particle Stochastic Approximationcited by 6
  18. 2021Relative Entropy Gradient Sampler for Unnormalized Distributionscited by 5
  19. 2023Efficient displacement convex optimization with particle gradient descentcited by 5
  20. 2021Multilevel Stein variational gradient descent with applications to Bayesian inverse problemscited by 4
  21. 2022A stochastic Stein Variational Newton methodcited by 4
  22. 2022A Note on the Convergence of Mirrored Stein Variational Gradient Descent under (L0,L1)-Smoothness Conditioncited by 4
  23. 2024Wasserstein Gradient Boosting: A Framework for Distribution-Valued Supervised Learningcited by 4
  24. 2021Loss function based second-order Jensen inequality and its application to particle variational inferencecited by 3
  25. 2021Stein Variational Gradient Descent with Multiple Kernelcited by 3
  26. 2022Online, Informative MCMC Thinning with Kernelized Stein Discrepancycited by 3
  27. 2023Learning Rate Free Sampling in Constrained Domainscited by 3
  28. 2024A Particle Algorithm for Mean-Field Variational Inferencecited by 3
  29. 2026Finite-Particle Rates for Regularized Stein Variational Gradient Descentcited by 3
  30. 2026Sobolev Regularized MMD Gradient Flowcited by 3
  31. 2021Low-rank variational Bayes correction to the Laplace methodcited by 2
  32. 2022Improved Stein Variational Gradient Descent with Importance Weightscited by 2
  33. 2024Stein Boltzmann Sampling: A Variational Approach for Global Optimizationcited by 2
  34. 2024Long-time asymptotics of noisy SVGD outside the population limitcited by 2
  35. 2025Stationary MMD Pointscited by 2
  36. 2025The Minimax Lower Bound of Kernel Stein Discrepancy Estimationcited by 2
  37. 2026Dual-space posterior sampling for Bayesian inference in constrained inverse problemscited by 2
  38. 2026Minimax Lower Bounds of Kernel Discrepancy Estimation: MMD, HSIC, KSDcited by 2

50 most cited of 61 All 61 (JSON)