# Projected Wasserstein gradient descent for high-dimensional Bayesian inference

arXiv: https://arxiv.org/abs/2102.06350 · HTML: https://mlsanity.com/paper/2102.06350/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 51 references that are in the map.
- First posted on arXiv 12 Feb 2021.

## Builds on (most cited first, 23 of 23)

- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [Stein Variational Gradient Descent as Gradient Flow](https://mlsanity.com/paper/1704.07520/) (2017)
- [Interacting particle solutions of Fokker-Planck equations through gradient-log-density estimation](https://mlsanity.com/paper/2006.00702/) (2020)
- [On the geometry of Stein variational gradient descent](https://mlsanity.com/paper/1912.00894/) (2019)
- [A Stein variational Newton method](https://mlsanity.com/paper/1806.03085/) (2018)
- [Derivative-Informed Projected Neural Networks for High-Dimensional Parametric Maps Governed by PDEs](https://mlsanity.com/paper/2011.15110/) (2020)
- [Accelerated Information Gradient flow](https://mlsanity.com/paper/1909.02102/) (2019)
- [Dimension-independent likelihood-informed MCMC](https://mlsanity.com/paper/1411.3688/) (2014)
- [Projected Stein Variational Gradient Descent](https://mlsanity.com/paper/2002.03469/) (2020)
- [A computational framework for infinite-dimensional Bayesian inverse problems. Part I: The linearized case, with application to global seismic inversion](https://mlsanity.com/paper/1308.1313/) (2013)
- [Certified dimension reduction in nonlinear Bayesian inverse problems](https://mlsanity.com/paper/1807.03712/) (2018)
- [Information Newton's flow: second-order optimization method in probability space](https://mlsanity.com/paper/2001.04341/) (2020)
- [Projected Stein Variational Newton: A Fast and Scalable Bayesian Inference Method in High Dimensions](https://mlsanity.com/paper/1901.08659/) (2019)
- [Is There an Analog of Nesterov Acceleration for MCMC?](https://mlsanity.com/paper/1902.00996/) (2019)
- [A Fast and Scalable Method for A-Optimal Design of Experiments for Infinite-dimensional Bayesian Nonlinear Inverse Problems](https://mlsanity.com/paper/1410.5899/) (2014)
- [Stochastic Gradient MCMC with Stale Gradients](https://mlsanity.com/paper/1610.06664/) (2016)
- [A fast and scalable computational framework for large-scale and high-dimensional Bayesian optimal experimental design](https://mlsanity.com/paper/2010.15196/) (2020)
- [Nonreversible Langevin Samplers: Splitting Schemes, Analysis and Implementation](https://mlsanity.com/paper/1701.04247/) (2017)
- [Optimal design of acoustic metamaterial cloaks under uncertainty](https://mlsanity.com/paper/2007.13252/) (2020)
- [Taylor approximation and variance reduction for PDE-constrained optimal control under uncertainty](https://mlsanity.com/paper/1804.04301/) (2018)
- [Mean-variance risk-averse optimal control of systems governed by PDEs with random parameter fields using quadratic approximations](https://mlsanity.com/paper/1602.07592/) (2016)
- [A-optimal encoding weights for nonlinear inverse problems, with applications to the Helmholtz inverse problem](https://mlsanity.com/paper/1612.02358/) (2016)
- [Bayesian inference of heterogeneous epidemic models: Application to COVID-19 spread accounting for long-term care facilities](https://mlsanity.com/paper/2011.01058/) (2020)

## Built on by (most cited first, 24 of 24)

- [Repulsive Deep Ensembles are Bayesian](https://mlsanity.com/paper/2106.11642/) (2021)
- [Optimizing Functionals on the Space of Probabilities with Input Convex Neural Networks](https://mlsanity.com/paper/2106.00774/) (2021)
- [Forward-backward Gaussian variational inference via JKO in the Bures-Wasserstein Space](https://mlsanity.com/paper/2304.05398/) (2023)
- [Efficient Gradient Flows in Sliced-Wasserstein Space](https://mlsanity.com/paper/2110.10972/) (2021)
- [Residual-based error correction for neural operator accelerated infinite-dimensional Bayesian inverse problems](https://mlsanity.com/paper/2210.03008/) (2022)
- [Particle-based Variational Inference with Preconditioned Functional Gradient Flow](https://mlsanity.com/paper/2211.13954/) (2022)
- [Sampling via Gradient Flows in the Space of Probability Measures](https://mlsanity.com/paper/2310.03597/) (2023)
- [Gradient Flows for Sampling: Mean-Field Models, Gaussian Approximations and Affine Invariance](https://mlsanity.com/paper/2302.11024/) (2023)
- [Towards Understanding the Dynamics of Gaussian-Stein Variational Gradient Descent](https://mlsanity.com/paper/2305.14076/) (2023)
- [Optimal Neural Network Approximation of Wasserstein Gradient Direction via Convex Optimization](https://mlsanity.com/paper/2205.13098/) (2022)
- [Feature Space Particle Inference for Neural Network Ensembles](https://mlsanity.com/paper/2206.00944/) (2022)
- [The Score-Difference Flow for Implicit Generative Modeling](https://mlsanity.com/paper/2304.12906/) (2023)
- [Minimizing f-Divergences by Interpolating Velocity Fields](https://mlsanity.com/paper/2305.15577/) (2023)
- [Path-Guided Particle-based Sampling](https://mlsanity.com/paper/2412.03312/) (2024)
- [Visual Prompting for One-shot Controllable Video Editing without Inversion](https://mlsanity.com/paper/2504.14335/) (2025)
- [Input-gradient space particle inference for neural network ensembles](https://mlsanity.com/paper/2306.02775/) (2023)
- [Wasserstein Gradient Boosting: A Framework for Distribution-Valued Supervised Learning](https://mlsanity.com/paper/2405.09536/) (2024)
- [Forward-Euler time-discretization for Wasserstein gradient flows can be wrong](https://mlsanity.com/paper/2406.08209/) (2024)
- [A Library of Mirrors: Deep Neural Nets in Low Dimensions are Convex Lasso Models with Reflection Features](https://mlsanity.com/paper/2403.01046/) (2024)
- [Sequential Bayesian Design for Efficient Surrogate Construction in the Inversion of Darcy Flows](https://mlsanity.com/paper/2507.17713/) (2025)
- [Radon--Wasserstein Gradient Flows for Interacting-Particle Sampling in High Dimensions](https://mlsanity.com/paper/2602.05227/) (2026)
- [Liouville PDE-based sliced-Wasserstein flow](https://mlsanity.com/paper/2505.17204/) (2025)
- [Nesterov acceleration in optimizing over probability measures](https://mlsanity.com/paper/2607.23008/) (2026)
- [Wasserstein Gradient Flows and Forward-Only Diffusion Are Not Enough for Multimodal Sampling](https://mlsanity.com/paper/2610.02081/) (2026)

## Abstract (arXiv)

We propose a projected Wasserstein gradient descent method (pWGD) for high-dimensional Bayesian inference problems. The underlying density function of a particle system of WGD is approximated by kernel density estimation (KDE), which faces the long-standing curse of dimensionality. We overcome this challenge by exploiting the intrinsic low-rank structure in the difference between the posterior and prior distributions. The parameters are projected into a low-dimensional subspace to alleviate the approximation error of KDE in high dimensions. We formulate a projected Wasserstein gradient flow and analyze its convergence property under mild assumptions. Several numerical experiments illustrate the accuracy, convergence, and complexity scalability of pWGD with respect to parameter dimension, sample size, and processor cores.