# Regularized Stein Variational Gradient Flow

arXiv: https://arxiv.org/abs/2211.07861 · HTML: https://mlsanity.com/paper/2211.07861/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 53 references that are in the map.
- First posted on arXiv 15 Nov 2022.

## Builds on (most cited first, 22 of 22)

- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluation](https://mlsanity.com/paper/1602.03253/) (2016)
- [Rapid Convergence of the Unadjusted Langevin Algorithm: Isoperimetry Suffices](https://mlsanity.com/paper/1903.08568/) (2019)
- [Generalization Properties of Learning with Random Features](https://mlsanity.com/paper/1602.04474/) (2016)
- [Stein Variational Gradient Descent as Gradient Flow](https://mlsanity.com/paper/1704.07520/) (2017)
- [A Kernel Test of Goodness of Fit](https://mlsanity.com/paper/1602.02964/) (2016)
- [Sampling as optimization in the space of measures: The Langevin dynamics as a composite optimization problem](https://mlsanity.com/paper/1802.08089/) (2018)
- [Measuring Sample Quality with Kernels](https://mlsanity.com/paper/1703.01717/) (2017)
- [On the geometry of Stein variational gradient descent](https://mlsanity.com/paper/1912.00894/) (2019)
- [Deep Neural Tangent Kernel and Laplace Kernel Have the Same RKHS](https://mlsanity.com/paper/2009.10683/) (2020)
- [Improved analysis for a proximal algorithm for sampling](https://mlsanity.com/paper/2202.06386/) (2022)
- [SVGD as a kernelized Wasserstein gradient flow of the chi-squared divergence](https://mlsanity.com/paper/2006.02509/) (2020)
- [Towards a Theory of Non-Log-Concave Sampling: First-Order Stationarity Guarantees for Langevin Monte Carlo](https://mlsanity.com/paper/2202.05214/) (2022)
- [A Non-Asymptotic Analysis for Stein Variational Gradient Descent](https://mlsanity.com/paper/2006.09797/) (2020)
- [Stein Variational Policy Gradient](https://mlsanity.com/paper/1704.02399/) (2017)
- [Metrizing Weak Convergence with Maximum Mean Discrepancies](https://mlsanity.com/paper/2006.09268/) (2020)
- [Scaling limit of the Stein variational gradient descent: the mean field regime](https://mlsanity.com/paper/1805.04035/) (2018)
- [Stein Variational Gradient Descent With Matrix-Valued Kernels](https://mlsanity.com/paper/1910.12794/) (2019)
- [Randomized sketches for kernels: Fast and optimal non-parametric regression](https://mlsanity.com/paper/1501.06195/) (2015)
- [Measuring Sample Quality with Diffusions](https://mlsanity.com/paper/1611.06972/) (2016)
- [A Convergence Theory for SVGD in the Population Limit under Talagrand's Inequality T1](https://mlsanity.com/paper/2106.03076/) (2021)
- [On the Optimality of Kernel-Embedding Based Goodness-of-Fit Tests](https://mlsanity.com/paper/1709.08148/) (2017)

## Built on by (most cited first, 19 of 19)

- [Forward-backward Gaussian variational inference via JKO in the Bures-Wasserstein Space](https://mlsanity.com/paper/2304.05398/) (2023)
- [Zeroth-Order Sampling Methods for Non-Log-Concave Distributions: Alleviating Metastability by Denoising Diffusion](https://mlsanity.com/paper/2402.17886/) (2024)
- [Improved Finite-Particle Convergence Rates for Stein Variational Gradient Descent](https://mlsanity.com/paper/2409.08469/) (2024)
- [Towards Understanding the Dynamics of Gaussian-Stein Variational Gradient Descent](https://mlsanity.com/paper/2305.14076/) (2023)
- [Stein transport for Bayesian inference](https://mlsanity.com/paper/2409.01464/) (2024)
- [Kernel Approximation of Fisher-Rao Gradient Flows](https://mlsanity.com/paper/2410.20622/) (2024)
- [Sobolev Regularized MMD Gradient Flow](https://mlsanity.com/paper/2605.11884/) (2026)
- [Finite-Particle Rates for Regularized Stein Variational Gradient Descent](https://mlsanity.com/paper/2602.05172/) (2026)
- [Wasserstein Gradient Boosting: A Framework for Distribution-Valued Supervised Learning](https://mlsanity.com/paper/2405.09536/) (2024)
- [Inclusive KL Gradient Flows: Otto-Wasserstein, Fisher-Rao-Gaussian, and Local-Estimator Dynamics](https://mlsanity.com/paper/2411.00214/) (2024)
- [Stationary MMD Points](https://mlsanity.com/paper/2505.20754/) (2025)
- [Hessian-guided Perturbed Wasserstein Gradient Flows for Escaping Saddle Points](https://mlsanity.com/paper/2509.16974/) (2025)
- [To discretize continually: Mean shift interacting particle systems for Bayesian inference](https://mlsanity.com/paper/2605.14142/) (2026)
- [Radon--Wasserstein Gradient Flows for Interacting-Particle Sampling in High Dimensions](https://mlsanity.com/paper/2602.05227/) (2026)
- [Posterior sampling with CNN-based, Plug-and-Play regularization with applications to Post-Stack Seismic Inversion](https://mlsanity.com/paper/2212.14595/) (2022)
- [Complexity of Non-Log-Concave Sampling in Fisher Information](https://mlsanity.com/paper/2605.15859/) (2026)
- [Stability of Finite-Batch Particle Mean-Field Variational Inference Beyond Strong Convexity](https://mlsanity.com/paper/2608.11486/) (2026)
- [A Distributional Optimisation Perspective on Combining Models in Deep Learning](https://mlsanity.com/paper/2609.24328/) (2026)
- [Markov Chain Convergence in Kernel-Induced Geometry](https://mlsanity.com/paper/2610.13916/) (2026)

## Abstract (arXiv)

The Stein Variational Gradient Descent (SVGD) algorithm is a deterministic particle method for sampling. However, a mean-field analysis reveals that the gradient flow corresponding to the SVGD algorithm (i.e., the Stein Variational Gradient Flow) only provides a constant-order approximation to the Wasserstein Gradient Flow corresponding to the KL-divergence minimization. In this work, we propose the Regularized Stein Variational Gradient Flow, which interpolates between the Stein Variational Gradient Flow and the Wasserstein Gradient Flow. We establish various theoretical properties of the Regularized Stein Variational Gradient Flow (and its time-discretization) including convergence to equilibrium, existence and uniqueness of weak solutions, and stability of the solutions. We provide preliminary numerical evidence of the improved performance offered by the regularization.