# Stein Variational Gradient Descent With Matrix-Valued Kernels

arXiv: https://arxiv.org/abs/1910.12794 · HTML: https://mlsanity.com/paper/1910.12794/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Oct 2019.

## Built on by (most cited first, 30 of 32)

- [Robust Generalised Bayesian Inference for Intractable Likelihoods](https://mlsanity.com/paper/2104.07359/) (2021)
- [On Stein Variational Neural Network Ensembles](https://mlsanity.com/paper/2106.10760/) (2021)
- [Particle-based Variational Inference with Preconditioned Functional Gradient Flow](https://mlsanity.com/paper/2211.13954/) (2022)
- [Regularized Stein Variational Gradient Flow](https://mlsanity.com/paper/2211.07861/) (2022)
- [Sampling with Mirrored Stein Operators](https://mlsanity.com/paper/2106.12506/) (2021)
- [Convergence of Stein Variational Gradient Descent under a Weaker Smoothness Condition](https://mlsanity.com/paper/2206.00508/) (2022)
- [Towards Understanding the Dynamics of Gaussian-Stein Variational Gradient Descent](https://mlsanity.com/paper/2305.14076/) (2023)
- [Learning Equivariant Energy Based Models with Equivariant Stein Variational Gradient Descent](https://mlsanity.com/paper/2106.07832/) (2021)
- [Coin Sampling: Gradient-Based Bayesian Inference without Learning Rates](https://mlsanity.com/paper/2301.11294/) (2023)
- [Constrained Stein Variational Trajectory Optimization](https://mlsanity.com/paper/2308.12110/) (2023)
- [Noise-Free Sampling Algorithms via Regularized Wasserstein Proximals](https://mlsanity.com/paper/2308.14945/) (2023)
- [Stein ICP for Uncertainty Estimation in Point Cloud Matching](https://mlsanity.com/paper/2106.03287/) (2021)
- [Stein Variational Gradient Descent with Multiple Kernel](https://mlsanity.com/paper/2107.09338/) (2021)
- [A Note on the Convergence of Mirrored Stein Variational Gradient Descent under (L0,L1)-Smoothness Condition](https://mlsanity.com/paper/2206.09709/) (2022)
- [Improved Stein Variational Gradient Descent with Importance Weights](https://mlsanity.com/paper/2210.00462/) (2022)
- [Electrostatics-based particle sampling and approximate inference](https://mlsanity.com/paper/2406.20044/) (2024)
- [Stein Variational Evolution Strategies](https://mlsanity.com/paper/2410.10390/) (2024)
- [Dual-space posterior sampling for Bayesian inference in constrained inverse problems](https://mlsanity.com/paper/2603.00393/) (2026)
- [Tractable Computation of Expected Kernels](https://mlsanity.com/paper/2102.10562/) (2021)
- [Uncertainty Quantification of Graph Convolution Neural Network Models of Evolving Processes](https://mlsanity.com/paper/2402.11179/) (2024)
- [ELBOing Stein: Variational Bayes with Stein Mixture Inference](https://mlsanity.com/paper/2410.22948/) (2024)
- [Stein Diffusion Guidance: Training-Free Posterior Correction for Sampling Beyond High-Density Regions](https://mlsanity.com/paper/2507.05482/) (2025)
- [Soft Quality-Diversity Optimization](https://mlsanity.com/paper/2512.00810/) (2025)
- [Riesz-Kernel Stein Variational Gradient Descent: Renormalized Entropy and Long-Time Particle Limits](https://mlsanity.com/paper/2607.14527/) (2026)
- [Further analysis of multilevel Stein variational gradient descent with an application to the Bayesian inference of glacier ice models](https://mlsanity.com/paper/2212.03366/) (2022)
- [Posterior sampling with CNN-based, Plug-and-Play regularization with applications to Post-Stack Seismic Inversion](https://mlsanity.com/paper/2212.14595/) (2022)
- [Stein Random Feature Regression](https://mlsanity.com/paper/2406.00438/) (2024)
- [Accelerating Particle-based Energetic Variational Inference](https://mlsanity.com/paper/2504.03158/) (2025)
- [Adaptive Kernel Selection for Stein Variational Gradient Descent](https://mlsanity.com/paper/2510.02067/) (2025)
- [Accelerated Regularized Wasserstein Proximal Sampling Algorithms](https://mlsanity.com/paper/2601.09848/) (2026)

## Abstract (arXiv)

Stein variational gradient descent (SVGD) is a particle-based inference algorithm that leverages gradient information for efficient approximate inference. In this work, we enhance SVGD by leveraging preconditioning matrices, such as the Hessian and Fisher information matrix, to incorporate geometric information into SVGD updates. We achieve this by presenting a generalization of SVGD that replaces the scalar-valued kernels in vanilla SVGD with more general matrix-valued kernels. This yields a significant extension of SVGD, and more importantly, allows us to flexibly incorporate various preconditioning matrices to accelerate the exploration in the probability landscape. Empirical results show that our method outperforms vanilla SVGD and a variety of baseline approaches over a range of real-world Bayesian inference tasks.