# Augmented Message Passing Stein Variational Gradient Descent

arXiv: https://arxiv.org/abs/2305.10636 · HTML: https://mlsanity.com/paper/2305.10636/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 26 references that are in the map.
- First posted on arXiv 18 May 2023.

## Builds on (most cited first, 12 of 12)

- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [Bayesian Model-Agnostic Meta-Learning](https://mlsanity.com/paper/1806.03836/) (2018)
- [Stein Variational Gradient Descent as Gradient Flow](https://mlsanity.com/paper/1704.07520/) (2017)
- [Measuring Sample Quality with Kernels](https://mlsanity.com/paper/1703.01717/) (2017)
- [Protein structure generation via folding diffusion](https://mlsanity.com/paper/2209.15611/) (2022)
- [SVGD as a kernelized Wasserstein gradient flow of the chi-squared divergence](https://mlsanity.com/paper/2006.02509/) (2020)
- [A Stein variational Newton method](https://mlsanity.com/paper/1806.03085/) (2018)
- [Message Passing Stein Variational Gradient Descent](https://mlsanity.com/paper/1711.04425/) (2017)
- [Projected Stein Variational Gradient Descent](https://mlsanity.com/paper/2002.03469/) (2020)
- [Sliced Kernelized Stein Discrepancy](https://mlsanity.com/paper/2006.16531/) (2020)
- [A Convergence Theory for SVGD in the Population Limit under Talagrand's Inequality T1](https://mlsanity.com/paper/2106.03076/) (2021)
- [Grassmann Stein Variational Gradient Descent](https://mlsanity.com/paper/2202.03297/) (2022)

## Abstract (arXiv)

Stein Variational Gradient Descent (SVGD) is a popular particle-based method for Bayesian inference. However, its convergence suffers from the variance collapse, which reduces the accuracy and diversity of the estimation. In this paper, we study the isotropy property of finite particles during the convergence process and show that SVGD of finite particles cannot spread across the entire sample space. Instead, all particles tend to cluster around the particle center within a certain range and we provide an analytical bound for this cluster. To further improve the effectiveness of SVGD for high-dimensional problems, we propose the Augmented Message Passing SVGD (AUMP-SVGD) method, which is a two-stage optimization procedure that does not require sparsity of the target distribution, unlike the MP-SVGD method. Our algorithm achieves satisfactory accuracy and overcomes the variance collapse problem in various benchmark problems.