# A Convergence Theory for SVGD in the Population Limit under Talagrand's Inequality T1

arXiv: https://arxiv.org/abs/2106.03076 · HTML: https://mlsanity.com/paper/2106.03076/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 44 references that are in the map.
- First posted on arXiv 6 Jun 2021.

## Builds on (most cited first, 28 of 28)

- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluation](https://mlsanity.com/paper/1602.03253/) (2016)
- [Rapid Convergence of the Unadjusted Langevin Algorithm: Isoperimetry Suffices](https://mlsanity.com/paper/1903.08568/) (2019)
- [Theoretical guarantees for approximate sampling from smooth and log-concave densities](https://mlsanity.com/paper/1412.7392/) (2014)
- [Non-asymptotic convergence analysis for the Unadjusted Langevin Algorithm](https://mlsanity.com/paper/1507.05021/) (2015)
- [Stein Variational Gradient Descent as Gradient Flow](https://mlsanity.com/paper/1704.07520/) (2017)
- [A Kernel Test of Goodness of Fit](https://mlsanity.com/paper/1602.02964/) (2016)
- [Underdamped Langevin MCMC: A non-asymptotic analysis](https://mlsanity.com/paper/1707.03663/) (2017)
- [Analysis of Langevin Monte Carlo via convex optimization](https://mlsanity.com/paper/1802.09188/) (2018)
- [Sampling as optimization in the space of measures: The Langevin dynamics as a composite optimization problem](https://mlsanity.com/paper/1802.08089/) (2018)
- [Measuring Sample Quality with Kernels](https://mlsanity.com/paper/1703.01717/) (2017)
- [Efficient Bayesian computation by proximal Markov chain Monte Carlo: when Langevin meets Moreau](https://mlsanity.com/paper/1612.07471/) (2016)
- [On the geometry of Stein variational gradient descent](https://mlsanity.com/paper/1912.00894/) (2019)
- [The Randomized Midpoint Method for Log-Concave Sampling](https://mlsanity.com/paper/1909.05503/) (2019)
- [SVGD as a kernelized Wasserstein gradient flow of the chi-squared divergence](https://mlsanity.com/paper/2006.02509/) (2020)
- [Mirrored Langevin Dynamics](https://mlsanity.com/paper/1802.10174/) (2018)
- [A Non-Asymptotic Analysis for Stein Variational Gradient Descent](https://mlsanity.com/paper/2006.09797/) (2020)
- [Sampling from a log-concave distribution with Projected Langevin Monte Carlo](https://mlsanity.com/paper/1507.02564/) (2015)
- [Stein Variational Policy Gradient](https://mlsanity.com/paper/1704.02399/) (2017)
- [Scaling limit of the Stein variational gradient descent: the mean field regime](https://mlsanity.com/paper/1805.04035/) (2018)
- [Primal Dual Interpretation of the Proximal Stochastic Gradient Langevin Algorithm](https://mlsanity.com/paper/2006.09270/) (2020)
- [Federated Generalized Bayesian Learning via Distributed Stein Variational Gradient Descent](https://mlsanity.com/paper/2009.06419/) (2020)
- [Stochastic Stein Discrepancies](https://mlsanity.com/paper/2007.02857/) (2020)
- [Is There an Analog of Nesterov Acceleration for MCMC?](https://mlsanity.com/paper/1902.00996/) (2019)
- [Stein Variational Gradient Descent: many-particle and long-time asymptotics](https://mlsanity.com/paper/2102.12956/) (2021)
- [Fractional Langevin Monte Carlo: Exploring L'evy Driven Stochastic Differential Equations for Markov Chain Monte Carlo](https://mlsanity.com/paper/1706.03649/) (2017)
- [Scalable Thompson Sampling via Optimal Transport](https://mlsanity.com/paper/1902.07239/) (2019)
- [Learning Structural Weight Uncertainty for Sequential Decision-Making](https://mlsanity.com/paper/1801.00085/) (2017)

## Built on by (most cited first, 21 of 21)

- [Forward-backward Gaussian variational inference via JKO in the Bures-Wasserstein Space](https://mlsanity.com/paper/2304.05398/) (2023)
- [A Finite-Particle Convergence Rate for Stein Variational Gradient Descent](https://mlsanity.com/paper/2211.09721/) (2022)
- [Regularized Stein Variational Gradient Flow](https://mlsanity.com/paper/2211.07861/) (2022)
- [Zeroth-Order Sampling Methods for Non-Log-Concave Distributions: Alleviating Metastability by Denoising Diffusion](https://mlsanity.com/paper/2402.17886/) (2024)
- [Sampling with Mollified Interaction Energy Descent](https://mlsanity.com/paper/2210.13400/) (2022)
- [Improved Finite-Particle Convergence Rates for Stein Variational Gradient Descent](https://mlsanity.com/paper/2409.08469/) (2024)
- [Towards Understanding the Dynamics of Gaussian-Stein Variational Gradient Descent](https://mlsanity.com/paper/2305.14076/) (2023)
- [Tuning-Free Maximum Likelihood Training of Latent Variable Models via Coin Betting](https://mlsanity.com/paper/2305.14916/) (2023)
- [Coin Sampling: Gradient-Based Bayesian Inference without Learning Rates](https://mlsanity.com/paper/2301.11294/) (2023)
- [Provably Fast Finite Particle Variants of SVGD via Virtual Particle Stochastic Approximation](https://mlsanity.com/paper/2305.17558/) (2023)
- [Towards a Better Theoretical Understanding of Independent Subnetwork Training](https://mlsanity.com/paper/2306.16484/) (2023)
- [Finite-Particle Rates for Regularized Stein Variational Gradient Descent](https://mlsanity.com/paper/2602.05172/) (2026)
- [Learning Rate Free Sampling in Constrained Domains](https://mlsanity.com/paper/2305.14943/) (2023)
- [Bayesian Multi-Task Transfer Learning for Soft Prompt Tuning](https://mlsanity.com/paper/2402.08594/) (2024)
- [Long-time asymptotics of noisy SVGD outside the population limit](https://mlsanity.com/paper/2406.11929/) (2024)
- [Quantitative Local Convergence of Mean-Field Stein Variational Gradient Flow](https://mlsanity.com/paper/2605.09456/) (2026)
- [Gradient Flow Sampler-based Distributionally Robust Optimization](https://mlsanity.com/paper/2510.25956/) (2025)
- [To discretize continually: Mean shift interacting particle systems for Bayesian inference](https://mlsanity.com/paper/2605.14142/) (2026)
- [Augmented Message Passing Stein Variational Gradient Descent](https://mlsanity.com/paper/2305.10636/) (2023)
- [Adaptive Kernel Selection for Stein Variational Gradient Descent](https://mlsanity.com/paper/2510.02067/) (2025)
- [Generative Modeling through Koopman Spectral Analysis: An Operator-Theoretic Perspective](https://mlsanity.com/paper/2512.18837/) (2025)

## Abstract (arXiv)

Stein Variational Gradient Descent (SVGD) is an algorithm for sampling from a target density which is known up to a multiplicative constant. Although SVGD is a popular algorithm in practice, its theoretical study is limited to a few recent works. We study the convergence of SVGD in the population limit, (i.e., with an infinite number of particles) to sample from a non-logconcave target distribution satisfying Talagrand's inequality T1. We first establish the convergence of the algorithm. Then, we establish a dimension-dependent complexity bound in terms of the Kernelized Stein Discrepancy (KSD). Unlike existing works, we do not assume that the KSD is bounded along the trajectory of the algorithm. Our approach relies on interpreting SVGD as a gradient descent over a space of probability measures.