# Stein Variational Gradient Descent: many-particle and long-time asymptotics arXiv: https://arxiv.org/abs/2102.12956 ยท HTML: https://mlsanity.com/paper/2102.12956/ ## Key facts - Cited by 15 papers in the map (data through 14 Oct 2026). - Builds on 24 of its 90 references that are in the map. - First posted on arXiv 25 Feb 2021. ## Builds on (most cited first, 24 of 24) - [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016) - [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016) - [A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluation](https://mlsanity.com/paper/1602.03253/) (2016) - [Advances in Variational Inference](https://mlsanity.com/paper/1711.05597/) (2017) - [Stein Variational Gradient Descent as Gradient Flow](https://mlsanity.com/paper/1704.07520/) (2017) - [A Complete Recipe for Stochastic Gradient MCMC](https://mlsanity.com/paper/1506.04696/) (2015) - [A Kernel Test of Goodness of Fit](https://mlsanity.com/paper/1602.02964/) (2016) - [Maximum Mean Discrepancy Gradient Flow](https://mlsanity.com/paper/1906.04370/) (2019) - [Measuring Sample Quality with Kernels](https://mlsanity.com/paper/1703.01717/) (2017) - [Theoretical guarantees for sampling and inference in generative models with latent diffusions](https://mlsanity.com/paper/1903.01608/) (2019) - [Variational Sequential Monte Carlo](https://mlsanity.com/paper/1705.11140/) (2017) - [Filtering Variational Objectives](https://mlsanity.com/paper/1705.09279/) (2017) - [On the geometry of Stein variational gradient descent](https://mlsanity.com/paper/1912.00894/) (2019) - [Solving high-dimensional Hamilton-Jacobi-Bellman PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path space](https://mlsanity.com/paper/2005.05409/) (2020) - [Sliced-Wasserstein Flows: Nonparametric Generative Modeling via Optimal Transport and Diffusions](https://mlsanity.com/paper/1806.08141/) (2018) - [SVGD as a kernelized Wasserstein gradient flow of the chi-squared divergence](https://mlsanity.com/paper/2006.02509/) (2020) - [A Non-Asymptotic Analysis for Stein Variational Gradient Descent](https://mlsanity.com/paper/2006.09797/) (2020) - [A Unified Particle-Optimization Framework for Scalable Bayesian Sampling](https://mlsanity.com/paper/1805.11659/) (2018) - [Scaling limit of the Stein variational gradient descent: the mean field regime](https://mlsanity.com/paper/1805.04035/) (2018) - [A Contrastive Divergence for Combining Variational Inference and MCMC](https://mlsanity.com/paper/1905.04062/) (2019) - [Stochastic Gradient MCMC with Repulsive Forces](https://mlsanity.com/paper/1812.00071/) (2018) - [Stein Variational Gradient Descent as Moment Matching](https://mlsanity.com/paper/1810.11693/) (2018) - [Measure Transport with Kernel Stein Discrepancy](https://mlsanity.com/paper/2010.11779/) (2020) - [Wasserstein variational gradient descent: From semi-discrete optimal transport to ensemble variational inference](https://mlsanity.com/paper/1811.02827/) (2018) ## Built on by (most cited first, 15 of 15) - [A Convergence Theory for SVGD in the Population Limit under Talagrand's Inequality T1](https://mlsanity.com/paper/2106.03076/) (2021) - [Convergence of Stein Variational Gradient Descent under a Weaker Smoothness Condition](https://mlsanity.com/paper/2206.00508/) (2022) - [Stein transport for Bayesian inference](https://mlsanity.com/paper/2409.01464/) (2024) - [GAD-PVI: A General Accelerated Dynamic-Weight Particle-Based Variational Inference Framework](https://mlsanity.com/paper/2312.16429/) (2023) - [Provably Fast Finite Particle Variants of SVGD via Virtual Particle Stochastic Approximation](https://mlsanity.com/paper/2305.17558/) (2023) - [Accelerated Stein Variational Gradient Flow](https://mlsanity.com/paper/2503.23462/) (2025) - [Long-time asymptotics of noisy SVGD outside the population limit](https://mlsanity.com/paper/2406.11929/) (2024) - [Quantitative Local Convergence of Mean-Field Stein Variational Gradient Flow](https://mlsanity.com/paper/2605.09456/) (2026) - [Smoothed Distance Kernels for MMDs and Applications in Wasserstein Gradient Flows](https://mlsanity.com/paper/2504.07820/) (2025) - [Towards understanding Accelerated Stein Variational Gradient Flow -- Analysis of Generalized Bilinear Kernels for Gaussian target distributions](https://mlsanity.com/paper/2509.04008/) (2025) - [Radon--Wasserstein Gradient Flows for Interacting-Particle Sampling in High Dimensions](https://mlsanity.com/paper/2602.05227/) (2026) - [Riesz-Kernel Stein Variational Gradient Descent: Renormalized Entropy and Long-Time Particle Limits](https://mlsanity.com/paper/2607.14527/) (2026) - [Adaptive Kernel Selection for Stein Variational Gradient Descent](https://mlsanity.com/paper/2510.02067/) (2025) - [Well-Posed KL-Regularized Control via Wasserstein and Kalman-Wasserstein KL Divergences](https://mlsanity.com/paper/2602.02250/) (2026) - [Interacting particle guidance for sampling reward-tilted generative priors](https://mlsanity.com/paper/2609.37227/) (2026) ## Abstract (arXiv) Stein variational gradient descent (SVGD) refers to a class of methods for Bayesian inference based on interacting particle systems. In this paper, we consider the originally proposed deterministic dynamics as well as a stochastic variant, each of which represent one of the two main paradigms in Bayesian computational statistics: variational inference and Markov chain Monte Carlo. As it turns out, these are tightly linked through a correspondence between gradient flow structures and large-deviation principles rooted in statistical physics. To expose this relationship, we develop the cotangent space construction for the Stein geometry, prove its basic properties, and determine the large-deviation functional governing the many-particle limit for the empirical measure. Moreover, we identify the Stein-Fisher information (or kernelised Stein discrepancy) as its leading order contribution in the long-time and many-particle regime in the sense of Gamma-convergence, shedding some light on the finite-particle properties of SVGD. Finally, we establish a comparison principle between the Stein-Fisher information and RKHS-norms that might be of independent interest.