# Improved Finite-Particle Convergence Rates for Stein Variational Gradient Descent

arXiv: https://arxiv.org/abs/2409.08469 · HTML: https://mlsanity.com/paper/2409.08469/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 49 references that are in the map.
- First posted on arXiv 13 Sep 2024.

## Builds on (most cited first, 27 of 27)

- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Reinforcement Learning with Deep Energy-Based Policies](https://mlsanity.com/paper/1702.08165/) (2017)
- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysis](https://mlsanity.com/paper/1702.03849/) (2017)
- [A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluation](https://mlsanity.com/paper/1602.03253/) (2016)
- [Rapid Convergence of the Unadjusted Langevin Algorithm: Isoperimetry Suffices](https://mlsanity.com/paper/1903.08568/) (2019)
- [Stein Variational Gradient Descent as Gradient Flow](https://mlsanity.com/paper/1704.07520/) (2017)
- [A Kernel Test of Goodness of Fit](https://mlsanity.com/paper/1602.02964/) (2016)
- [Measuring Sample Quality with Kernels](https://mlsanity.com/paper/1703.01717/) (2017)
- [Measuring Sample Quality with Stein's Method](https://mlsanity.com/paper/1506.03039/) (2015)
- [On the geometry of Stein variational gradient descent](https://mlsanity.com/paper/1912.00894/) (2019)
- [SVGD as a kernelized Wasserstein gradient flow of the chi-squared divergence](https://mlsanity.com/paper/2006.02509/) (2020)
- [A Non-Asymptotic Analysis for Stein Variational Gradient Descent](https://mlsanity.com/paper/2006.09797/) (2020)
- [Scaling limit of the Stein variational gradient descent: the mean field regime](https://mlsanity.com/paper/1805.04035/) (2018)
- [Stochastic Stein Discrepancies](https://mlsanity.com/paper/2007.02857/) (2020)
- [Stein Variational Model Predictive Control](https://mlsanity.com/paper/2011.07641/) (2020)
- [Learning to Draw Samples with Amortized Stein Variational Gradient Descent](https://mlsanity.com/paper/1707.06626/) (2017)
- [A Finite-Particle Convergence Rate for Stein Variational Gradient Descent](https://mlsanity.com/paper/2211.09721/) (2022)
- [A Convergence Theory for SVGD in the Population Limit under Talagrand's Inequality T1](https://mlsanity.com/paper/2106.03076/) (2021)
- [Regularized Stein Variational Gradient Flow](https://mlsanity.com/paper/2211.07861/) (2022)
- [Convergence of Stein Variational Gradient Descent under a Weaker Smoothness Condition](https://mlsanity.com/paper/2206.00508/) (2022)
- [Targeted Separation and Convergence with Kernel Discrepancies](https://mlsanity.com/paper/2209.12835/) (2022)
- [Controlling Moments with Kernel Stein Discrepancies](https://mlsanity.com/paper/2211.05408/) (2022)
- [Towards Understanding the Dynamics of Gaussian-Stein Variational Gradient Descent](https://mlsanity.com/paper/2305.14076/) (2023)
- [Minimax Optimal Goodness-of-Fit Testing with Kernel Stein Discrepancy](https://mlsanity.com/paper/2404.08278/) (2024)
- [Provably Fast Finite Particle Variants of SVGD via Virtual Particle Stochastic Approximation](https://mlsanity.com/paper/2305.17558/) (2023)
- [Long-time asymptotics of noisy SVGD outside the population limit](https://mlsanity.com/paper/2406.11929/) (2024)

## Built on by (most cited first, 14 of 14)

- [Sobolev Regularized MMD Gradient Flow](https://mlsanity.com/paper/2605.11884/) (2026)
- [Finite-Particle Rates for Regularized Stein Variational Gradient Descent](https://mlsanity.com/paper/2602.05172/) (2026)
- [Finite-Particle Convergence Rates for Conservative and Non-Conservative Drifting Models](https://mlsanity.com/paper/2605.22795/) (2026)
- [Quantitative Local Convergence of Mean-Field Stein Variational Gradient Flow](https://mlsanity.com/paper/2605.09456/) (2026)
- [Minimax Estimation of Kernel Stein Discrepancy: Trace versus Hilbert-Schmidt Scales](https://mlsanity.com/paper/2607.03367/) (2026)
- [Stationary MMD Points](https://mlsanity.com/paper/2505.20754/) (2025)
- [To discretize continually: Mean shift interacting particle systems for Bayesian inference](https://mlsanity.com/paper/2605.14142/) (2026)
- [Beyond Propagation of Chaos: A Stochastic Algorithm for Mean Field Optimization](https://mlsanity.com/paper/2503.13115/) (2025)
- [Radon--Wasserstein Gradient Flows for Interacting-Particle Sampling in High Dimensions](https://mlsanity.com/paper/2602.05227/) (2026)
- [Riesz-Kernel Stein Variational Gradient Descent: Renormalized Entropy and Long-Time Particle Limits](https://mlsanity.com/paper/2607.14527/) (2026)
- [Branching Stein Variational Gradient Descent for sampling multimodal distributions](https://mlsanity.com/paper/2506.13916/) (2025)
- [Adaptive Kernel Selection for Stein Variational Gradient Descent](https://mlsanity.com/paper/2510.02067/) (2025)
- [Complexity of Non-Log-Concave Sampling in Fisher Information](https://mlsanity.com/paper/2605.15859/) (2026)
- [Quantitative Target Convergence and Uniform-in-Time Propagation of Chaos for Langevin-Regularized SVGD](https://mlsanity.com/paper/2608.28827/) (2026)

## Abstract (arXiv)

We provide finite-particle convergence rates for the Stein Variational Gradient Descent (SVGD) algorithm in the Kernelized Stein Discrepancy (mathsfKSD) and Wasserstein-2 metrics. Our key insight is that the time derivative of the relative entropy between the joint density of N particle locations and the N-fold product target measure, starting from a regular initial distribution, splits into a dominant `negative part' proportional to N times the expected mathsfKSD2 and a smaller `positive part'. This observation leads to mathsfKSD rates of order 1/√N, in both continuous and discrete time, providing a near optimal (in the sense of matching the corresponding i.i.d. rates) double exponential improvement over the recent result by Shi and Mackey (2024). Under mild assumptions on the kernel and potential, these bounds also grow polynomially in the dimension d. By adding a bilinear component to the kernel, the above approach is used to further obtain Wasserstein-2 convergence in continuous time. For the case of `bilinear + Mat'ern' kernels, we derive Wasserstein-2 rates that exhibit a curse-of-dimensionality similar to the i.i.d. setting. We also obtain marginal convergence and long-time propagation of chaos results for the time-averaged particle laws.