# A Unified Particle-Optimization Framework for Scalable Bayesian Sampling

arXiv: https://arxiv.org/abs/1805.11659 · HTML: https://mlsanity.com/paper/1805.11659/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 May 2018.

## Built on by (most cited first, 30 of 44)

- [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023)
- [Repulsive Deep Ensembles are Bayesian](https://mlsanity.com/paper/2106.11642/) (2021)
- [Is MC Dropout Bayesian?](https://mlsanity.com/paper/2110.04286/) (2021)
- [DreamFlow: High-Quality Text-to-3D Generation by Approximating Probability Flow](https://mlsanity.com/paper/2403.14966/) (2024)
- [Sqrt(d) Dimension Dependence of Langevin Monte Carlo](https://mlsanity.com/paper/2109.03839/) (2021)
- [Discrete Langevin Sampler via Wasserstein Gradient Flow](https://mlsanity.com/paper/2206.14897/) (2022)
- [Stein Variational Gradient Descent: many-particle and long-time asymptotics](https://mlsanity.com/paper/2102.12956/) (2021)
- [Retrieval-Augmented Score Distillation for Text-to-3D Generation](https://mlsanity.com/paper/2402.02972/) (2024)
- [Particle-based Variational Inference with Generalized Wasserstein Gradient Flow](https://mlsanity.com/paper/2310.16516/) (2023)
- [Feature Space Particle Inference for Neural Network Ensembles](https://mlsanity.com/paper/2206.00944/) (2022)
- [GAD-PVI: A General Accelerated Dynamic-Weight Particle-Based Variational Inference Framework](https://mlsanity.com/paper/2312.16429/) (2023)
- [RecDreamer: Consistent Text-to-3D Generation via Uniform Score Distillation](https://mlsanity.com/paper/2502.12640/) (2025)
- [Coin Sampling: Gradient-Based Bayesian Inference without Learning Rates](https://mlsanity.com/paper/2301.11294/) (2023)
- [Make Prompts Adaptable: Bayesian Modeling for Vision-Language Prompt Learning with Data-Dependent Prior](https://mlsanity.com/paper/2401.06799/) (2024)
- [Bridging the Sim-to-Real Gap with Bayesian Inference](https://mlsanity.com/paper/2403.16644/) (2024)
- [Function Space Diversity for Uncertainty Prediction via Repulsive Last-Layer Ensembles](https://mlsanity.com/paper/2412.15758/) (2024)
- [DPVI: A Dynamic-Weight Particle-Based Variational Inference Framework](https://mlsanity.com/paper/2112.00945/) (2021)
- [Particle-based Online Bayesian Sampling](https://mlsanity.com/paper/2302.14796/) (2023)
- [Input-gradient space particle inference for neural network ensembles](https://mlsanity.com/paper/2306.02775/) (2023)
- [Data-Efficient Task Generalization via Probabilistic Model-based Meta Reinforcement Learning](https://mlsanity.com/paper/2311.07558/) (2023)
- [Relative Entropy Gradient Sampler for Unnormalized Distributions](https://mlsanity.com/paper/2110.02787/) (2021)
- [nabla-Reasoner: LLM Reasoning via Test-Time Gradient Descent in Latent Space](https://mlsanity.com/paper/2603.04948/) (2026)
- [Multiscale Invertible Generative Networks for High-Dimensional Bayesian Inference](https://mlsanity.com/paper/2105.05489/) (2021)
- [On the Generative Utility of Cyclic Conditionals](https://mlsanity.com/paper/2106.15962/) (2021)
- [Stein Variational Gradient Descent with Multiple Kernel](https://mlsanity.com/paper/2107.09338/) (2021)
- [GradeADreamer: Enhanced Text-to-3D Generation Using Gaussian Splatting and Multi-View Diffusion](https://mlsanity.com/paper/2406.09850/) (2024)
- [Loss function based second-order Jensen inequality and its application to particle variational inference](https://mlsanity.com/paper/2106.05010/) (2021)
- [De-randomizing MCMC dynamics with the diffusion Stein operator](https://mlsanity.com/paper/2110.03768/) (2021)
- [A Deterministic Sampling Method via Maximum Mean Discrepancy Flow with Adaptive Kernel](https://mlsanity.com/paper/2111.10722/) (2021)
- [Rethinking the Diffusion Models for Numerical Tabular Data Imputation from the Perspective of Wasserstein Gradient Flow](https://mlsanity.com/paper/2406.15762/) (2024)

## Abstract (arXiv)

There has been recent interest in developing scalable Bayesian sampling methods such as stochastic gradient MCMC (SG-MCMC) and Stein variational gradient descent (SVGD) for big-data analysis. A standard SG-MCMC algorithm simulates samples from a discrete-time Markov chain to approximate a target distribution, thus samples could be highly correlated, an undesired property for SG-MCMC. In contrary, SVGD directly optimizes a set of particles to approximate a target distribution, and thus is able to obtain good approximations with relatively much fewer samples. In this paper, we propose a principle particle-optimization framework based on Wasserstein gradient flows to unify SG-MCMC and SVGD, and to allow new algorithms to be developed. Our framework interprets SG-MCMC as particle optimization on the space of probability measures, revealing a strong connection between SG-MCMC and SVGD. The key component of our framework is several particle-approximate techniques to efficiently solve the original partial differential equations on the space of probability measures. Extensive experiments on both synthetic data and deep neural networks demonstrate the effectiveness and efficiency of our framework for scalable Bayesian sampling.