# Projected Stein Variational Gradient Descent

arXiv: https://arxiv.org/abs/2002.03469 · HTML: https://mlsanity.com/paper/2002.03469/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Feb 2020.

## Built on by (most cited first, 30 of 33)

- [Derivative-Informed Neural Operator: An Efficient Framework for High-Dimensional Parametric Derivative Learning](https://mlsanity.com/paper/2206.10745/) (2022)
- [Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorial](https://mlsanity.com/paper/2305.04933/) (2023)
- [Projected Wasserstein gradient descent for high-dimensional Bayesian inference](https://mlsanity.com/paper/2102.06350/) (2021)
- [On Stein Variational Neural Network Ensembles](https://mlsanity.com/paper/2106.10760/) (2021)
- [Neural Variational Gradient Descent](https://mlsanity.com/paper/2107.10731/) (2021)
- [Derivative-enhanced Deep Operator Network](https://mlsanity.com/paper/2402.19242/) (2024)
- [Gradient Flows for Sampling: Mean-Field Models, Gaussian Approximations and Affine Invariance](https://mlsanity.com/paper/2302.11024/) (2023)
- [Towards Understanding the Dynamics of Gaussian-Stein Variational Gradient Descent](https://mlsanity.com/paper/2305.14076/) (2023)
- [Grassmann Stein Variational Gradient Descent](https://mlsanity.com/paper/2202.03297/) (2022)
- [Optimal Neural Network Approximation of Wasserstein Gradient Direction via Convex Optimization](https://mlsanity.com/paper/2205.13098/) (2022)
- [Feature Space Particle Inference for Neural Network Ensembles](https://mlsanity.com/paper/2206.00944/) (2022)
- [Coin Sampling: Gradient-Based Bayesian Inference without Learning Rates](https://mlsanity.com/paper/2301.11294/) (2023)
- [Path-Guided Particle-based Sampling](https://mlsanity.com/paper/2412.03312/) (2024)
- [Annealed Stein Variational Gradient Descent for Improved Uncertainty Estimation in Full-Waveform Inversion](https://mlsanity.com/paper/2410.13249/) (2024)
- [Are We Really Learning the Score Function? Reinterpreting Diffusion Models Through Wasserstein Gradient Flow Matching](https://mlsanity.com/paper/2509.00336/) (2025)
- [Multiscale Invertible Generative Networks for High-Dimensional Bayesian Inference](https://mlsanity.com/paper/2105.05489/) (2021)
- [Stein Variational Gradient Descent with Multiple Kernel](https://mlsanity.com/paper/2107.09338/) (2021)
- [A stochastic Stein Variational Newton method](https://mlsanity.com/paper/2204.09039/) (2022)
- [Improving the performance of Stein variational inference through extreme sparsification of physically-constrained neural network models](https://mlsanity.com/paper/2407.00761/) (2024)
- [Learning Rate Free Sampling in Constrained Domains](https://mlsanity.com/paper/2305.14943/) (2023)
- [Bayesian Multi-Task Transfer Learning for Soft Prompt Tuning](https://mlsanity.com/paper/2402.08594/) (2024)
- [Dimension reduction via score ratio matching](https://mlsanity.com/paper/2410.19990/) (2024)
- [Score Operator Newton transport](https://mlsanity.com/paper/2305.09792/) (2023)
- [Uncertainty Quantification of Graph Convolution Neural Network Models of Evolving Processes](https://mlsanity.com/paper/2402.11179/) (2024)
- [ESCORT: Efficient Stein-variational and Sliced Consistency-Optimized Temporal Belief Representation for POMDPs](https://mlsanity.com/paper/2510.21107/) (2025)
- [Radon--Wasserstein Gradient Flows for Interacting-Particle Sampling in High Dimensions](https://mlsanity.com/paper/2602.05227/) (2026)
- [Augmented Message Passing Stein Variational Gradient Descent](https://mlsanity.com/paper/2305.10636/) (2023)
- [Bayesian Low-Rank LeArning (Bella): A Practical Approach to Bayesian Neural Networks](https://mlsanity.com/paper/2407.20891/) (2024)
- [A Trust-Region Method for Graphical Stein Variational Inference](https://mlsanity.com/paper/2410.16195/) (2024)
- [Stein Variational Newton Neural Network Ensembles](https://mlsanity.com/paper/2411.01887/) (2024)

## Abstract (arXiv)

The curse of dimensionality is a longstanding challenge in Bayesian inference in high dimensions. In this work, we propose a projected Stein variational gradient descent (pSVGD) method to overcome this challenge by exploiting the fundamental property of intrinsic low dimensionality of the data informed subspace stemming from ill-posedness of such problems. We adaptively construct the subspace using a gradient information matrix of the log-likelihood, and apply pSVGD to the much lower-dimensional coefficients of the parameter projection. The method is demonstrated to be more accurate and efficient than SVGD. It is also shown to be more scalable with respect to the number of parameters, samples, data points, and processor cores via experiments with parameters dimensions ranging from the hundreds to the tens of thousands.