# Projected Stein Variational Newton: A Fast and Scalable Bayesian Inference Method in High Dimensions

arXiv: https://arxiv.org/abs/1901.08659 · HTML: https://mlsanity.com/paper/1901.08659/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jan 2019.

## Built on by (most cited first, 21 of 21)

- [Derivative-Informed Neural Operator: An Efficient Framework for High-Dimensional Parametric Derivative Learning](https://mlsanity.com/paper/2206.10745/) (2022)
- [Sampling in Unit Time with Kernel Fisher-Rao Flow](https://mlsanity.com/paper/2401.03892/) (2024)
- [Projected Wasserstein gradient descent for high-dimensional Bayesian inference](https://mlsanity.com/paper/2102.06350/) (2021)
- [Residual-based error correction for neural operator accelerated infinite-dimensional Bayesian inverse problems](https://mlsanity.com/paper/2210.03008/) (2022)
- [Particle-based Variational Inference with Preconditioned Functional Gradient Flow](https://mlsanity.com/paper/2211.13954/) (2022)
- [Sampling with Mirrored Stein Operators](https://mlsanity.com/paper/2106.12506/) (2021)
- [Adaptive operator learning for infinite-dimensional Bayesian inverse problems](https://mlsanity.com/paper/2310.17844/) (2023)
- [Multilevel Stein variational gradient descent with applications to Bayesian inverse problems](https://mlsanity.com/paper/2104.01945/) (2021)
- [Multiscale Invertible Generative Networks for High-Dimensional Bayesian Inference](https://mlsanity.com/paper/2105.05489/) (2021)
- [A stochastic Stein Variational Newton method](https://mlsanity.com/paper/2204.09039/) (2022)
- [Improving the performance of Stein variational inference through extreme sparsification of physically-constrained neural network models](https://mlsanity.com/paper/2407.00761/) (2024)
- [Active Slices for Sliced Stein Discrepancy](https://mlsanity.com/paper/2102.03159/) (2021)
- [A Deterministic Sampling Method via Maximum Mean Discrepancy Flow with Adaptive Kernel](https://mlsanity.com/paper/2111.10722/) (2021)
- [Stein Variational Evolution Strategies](https://mlsanity.com/paper/2410.10390/) (2024)
- [Dual-space posterior sampling for Bayesian inference in constrained inverse problems](https://mlsanity.com/paper/2603.00393/) (2026)
- [A Library of Mirrors: Deep Neural Nets in Low Dimensions are Convex Lasso Models with Reflection Features](https://mlsanity.com/paper/2403.01046/) (2024)
- [Soft Quality-Diversity Optimization](https://mlsanity.com/paper/2512.00810/) (2025)
- [Further analysis of multilevel Stein variational gradient descent with an application to the Bayesian inference of glacier ice models](https://mlsanity.com/paper/2212.03366/) (2022)
- [A Trust-Region Method for Graphical Stein Variational Inference](https://mlsanity.com/paper/2410.16195/) (2024)
- [Stein Variational Newton Neural Network Ensembles](https://mlsanity.com/paper/2411.01887/) (2024)
- [Accelerating Particle-based Energetic Variational Inference](https://mlsanity.com/paper/2504.03158/) (2025)

## Abstract (arXiv)

We propose a fast and scalable variational method for Bayesian inference in high-dimensional parameter space, which we call projected Stein variational Newton (pSVN) method. We exploit the intrinsic low-dimensional geometric structure of the posterior distribution in the high-dimensional parameter space via its Hessian (of the log posterior) operator and perform a parallel update of the parameter samples projected into a low-dimensional subspace by an SVN method. The subspace is adaptively constructed using the eigenvectors of the averaged Hessian at the current samples. We demonstrate fast convergence of the proposed method and its scalability with respect to the number of parameters, samples, and processor cores.