# Towards understanding Accelerated Stein Variational Gradient Flow -- Analysis of Generalized Bilinear Kernels for Gaussian target distributions arXiv: https://arxiv.org/abs/2509.04008 ยท HTML: https://mlsanity.com/paper/2509.04008/ ## Key facts - Cited by 2 papers in the map (data through 14 Oct 2026). - Builds on 30 of its 68 references that are in the map. - First posted on arXiv 4 Sep 2025. ## Builds on (most cited first, 30 of 30) - [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015) - [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016) - [Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks](https://mlsanity.com/paper/1502.05336/) (2015) - [A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights](https://mlsanity.com/paper/1503.01243/) (2015) - [Stein Variational Gradient Descent as Gradient Flow](https://mlsanity.com/paper/1704.07520/) (2017) - [A Variational Perspective on Accelerated Methods in Optimization](https://mlsanity.com/paper/1603.04245/) (2016) - [Bayesian imaging using Plug & Play priors: when Langevin meets Tweedie](https://mlsanity.com/paper/2103.04715/) (2021) - [Structured Logconcave Sampling with a Restricted Gaussian Oracle](https://mlsanity.com/paper/2010.03106/) (2020) - [On the geometry of Stein variational gradient descent](https://mlsanity.com/paper/1912.00894/) (2019) - [A unifying tutorial on Approximate Message Passing](https://mlsanity.com/paper/2105.02180/) (2021) - [Accelerated Information Gradient flow](https://mlsanity.com/paper/1909.02102/) (2019) - [Hessian Riemannian gradient flows in convex programming](https://mlsanity.com/paper/1811.10331/) (2018) - [Information Newton's flow: second-order optimization method in probability space](https://mlsanity.com/paper/2001.04341/) (2020) - [First-order optimization algorithms via inertial systems with Hessian driven damping](https://mlsanity.com/paper/1907.10536/) (2019) - [Gradient Flows for Sampling: Mean-Field Models, Gaussian Approximations and Affine Invariance](https://mlsanity.com/paper/2302.11024/) (2023) - [The rate of convergence of Nesterov's accelerated forward-backward method is actually faster than 1/k2](https://mlsanity.com/paper/1510.08740/) (2015) - [Accelerated Flow for Probability Distributions](https://mlsanity.com/paper/1901.03317/) (2019) - [Stein Variational Gradient Descent: many-particle and long-time asymptotics](https://mlsanity.com/paper/2102.12956/) (2021) - [Fast convex optimization via inertial dynamics with Hessian driven damping](https://mlsanity.com/paper/1601.07113/) (2016) - [Towards Understanding the Dynamics of Gaussian-Stein Variational Gradient Descent](https://mlsanity.com/paper/2305.14076/) (2023) - [Neural Sampling from Boltzmann Densities: Fisher-Rao Curves in the Wasserstein Geometry](https://mlsanity.com/paper/2410.03282/) (2024) - [Invertible Neural Networks versus MCMC for Posterior Reconstruction in Grazing Incidence X-Ray Fluorescence](https://mlsanity.com/paper/2102.03189/) (2021) - [Stein transport for Bayesian inference](https://mlsanity.com/paper/2409.01464/) (2024) - [Accelerating optimization over the space of probability measures](https://mlsanity.com/paper/2310.04006/) (2023) - [Telegrapher's Generative Model via Kac Flows](https://mlsanity.com/paper/2506.20641/) (2025) - [Accelerated Stein Variational Gradient Flow](https://mlsanity.com/paper/2503.23462/) (2025) - [Gradient-adjusted underdamped Langevin dynamics for sampling](https://mlsanity.com/paper/2410.08987/) (2024) - [A note on the unique properties of the Kullback--Leibler divergence for sampling via gradient flows](https://mlsanity.com/paper/2507.04330/) (2025) - [A qualitative difference between gradient flows of convex functions in finite- and infinite-dimensional Hilbert spaces](https://mlsanity.com/paper/2310.17610/) (2023) - [Polyak's Heavy Ball Method Achieves Accelerated Local Rate of Convergence under Polyak-Lojasiewicz Inequality](https://mlsanity.com/paper/2410.16849/) (2024) ## Built on by (most cited first, 2 of 2) - [SympFormer: Accelerated attention blocks via Inertial Dynamics on Density Manifolds](https://mlsanity.com/paper/2603.16535/) (2026) - [Well-Posed KL-Regularized Control via Wasserstein and Kalman-Wasserstein KL Divergences](https://mlsanity.com/paper/2602.02250/) (2026) ## Abstract (arXiv) Stein variational gradient descent (SVGD) is a kernel-based and non-parametric particle method for sampling from a target distribution, such as in Bayesian inference and other machine learning tasks. Different from other particle methods, SVGD does not require estimating the score, which is the gradient of the log-density. However, in practice, SVGD can be slow compared to score-estimation-based sampling algorithms. To design a fast and efficient high-dimensional sampling algorithm with the advantages of SVGD, we introduce accelerated SVGD (ASVGD), based on an accelerated gradient flow in a metric space of probability densities following Nesterov's method. We then derive a momentum-based discrete-time sampling algorithm, which evolves a set of particles deterministically. To stabilize the particles' position update, we also include a Wasserstein metric regularization. This paper extends the conference version citeSL2025. For the bilinear kernel and Gaussian target distributions, we study the kernel parameter and damping parameters with an optimal convergence rate of the proposed dynamics. This is achieved by analyzing the linearized accelerated gradient flows at the equilibrium. Interestingly, the optimal parameter is a constant, which does not depend on the covariance of the target distribution. For the generalized kernel functions, such as the Gaussian kernel, numerical examples with varied target distributions demonstrate the effectiveness of ASVGD compared to SVGD and other popular sampling methods. Furthermore, we show that in the setting of Bayesian neural networks, ASVGD outperforms SVGD significantly in terms of log-likelihood and total iteration times.