# Message Passing Stein Variational Gradient Descent

arXiv: https://arxiv.org/abs/1711.04425 · HTML: https://mlsanity.com/paper/1711.04425/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Nov 2017.

## Built on by (most cited first, 30 of 47)

- [Particle Guidance: non-I.I.D. Diverse Sampling with Diffusion Models](https://mlsanity.com/paper/2310.13102/) (2023)
- [Annealed Stein Variational Gradient Descent](https://mlsanity.com/paper/2101.09815/) (2021)
- [A Finite-Particle Convergence Rate for Stein Variational Gradient Descent](https://mlsanity.com/paper/2211.09721/) (2022)
- [On Stein Variational Neural Network Ensembles](https://mlsanity.com/paper/2106.10760/) (2021)
- [Particle-based Variational Inference with Preconditioned Functional Gradient Flow](https://mlsanity.com/paper/2211.13954/) (2022)
- [Neural Variational Gradient Descent](https://mlsanity.com/paper/2107.10731/) (2021)
- [Sampling with Mirrored Stein Operators](https://mlsanity.com/paper/2106.12506/) (2021)
- [Towards Understanding the Dynamics of Gaussian-Stein Variational Gradient Descent](https://mlsanity.com/paper/2305.14076/) (2023)
- [Grassmann Stein Variational Gradient Descent](https://mlsanity.com/paper/2202.03297/) (2022)
- [Variational Inference MPC using Tsallis Divergence](https://mlsanity.com/paper/2104.00241/) (2021)
- [Stein transport for Bayesian inference](https://mlsanity.com/paper/2409.01464/) (2024)
- [Learning Equivariant Energy Based Models with Equivariant Stein Variational Gradient Descent](https://mlsanity.com/paper/2106.07832/) (2021)
- [Coin Sampling: Gradient-Based Bayesian Inference without Learning Rates](https://mlsanity.com/paper/2301.11294/) (2023)
- [Neural Ensemble Search via Bayesian Sampling](https://mlsanity.com/paper/2109.02533/) (2021)
- [DPVI: A Dynamic-Weight Particle-Based Variational Inference Framework](https://mlsanity.com/paper/2112.00945/) (2021)
- [Provably Fast Finite Particle Variants of SVGD via Virtual Particle Stochastic Approximation](https://mlsanity.com/paper/2305.17558/) (2023)
- [Relative Entropy Gradient Sampler for Unnormalized Distributions](https://mlsanity.com/paper/2110.02787/) (2021)
- [Annealed Stein Variational Gradient Descent for Improved Uncertainty Estimation in Full-Waveform Inversion](https://mlsanity.com/paper/2410.13249/) (2024)
- [Localized Diffusion Models](https://mlsanity.com/paper/2505.04417/) (2025)
- [STAMP: Differentiable Task and Motion Planning via Stein Variational Gradient Descent](https://mlsanity.com/paper/2310.01775/) (2023)
- [Score-based deterministic density sampling](https://mlsanity.com/paper/2504.18130/) (2025)
- [Learning Rate Free Sampling in Constrained Domains](https://mlsanity.com/paper/2305.14943/) (2023)
- [Path Signatures for Diversity in Probabilistic Trajectory Optimisation](https://mlsanity.com/paper/2308.04071/) (2023)
- [Long-time asymptotics of noisy SVGD outside the population limit](https://mlsanity.com/paper/2406.11929/) (2024)
- [Localized Schr"odinger Bridge Sampler](https://mlsanity.com/paper/2409.07968/) (2024)
- [Stein's method for marginals on large graphical models](https://mlsanity.com/paper/2410.11771/) (2024)
- [Flow Matching Ergodic Coverage](https://mlsanity.com/paper/2504.17872/) (2025)
- [Low-rank variational Bayes correction to the Laplace method](https://mlsanity.com/paper/2111.12945/) (2021)
- [Gaussian Ensemble Belief Propagation for Efficient Inference in High-Dimensional Systems](https://mlsanity.com/paper/2402.08193/) (2024)
- [Stein Variational Evolution Strategies](https://mlsanity.com/paper/2410.10390/) (2024)

## Abstract (arXiv)

Stein variational gradient descent (SVGD) is a recently proposed particle-based Bayesian inference method, which has attracted a lot of interest due to its remarkable approximation ability and particle efficiency compared to traditional variational inference and Markov Chain Monte Carlo methods. However, we observed that particles of SVGD tend to collapse to modes of the target distribution, and this particle degeneracy phenomenon becomes more severe with higher dimensions. Our theoretical analysis finds out that there exists a negative correlation between the dimensionality and the repulsive force of SVGD which should be blamed for this phenomenon. We propose Message Passing SVGD (MP-SVGD) to solve this problem. By leveraging the conditional independence structure of probabilistic graphical models (PGMs), MP-SVGD converts the original high-dimensional global inference problem into a set of local ones over the Markov blanket with lower dimensions. Experimental results show its advantages of preventing vanishing repulsive force in high-dimensional space over SVGD, and its particle efficiency and approximation flexibility over other inference methods on graphical models.