# GAD-PVI: A General Accelerated Dynamic-Weight Particle-Based Variational Inference Framework

arXiv: https://arxiv.org/abs/2312.16429 · HTML: https://mlsanity.com/paper/2312.16429/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 81 references that are in the map.
- First posted on arXiv 27 Dec 2023.

## Builds on (most cited first, 30 of 35)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Computational Optimal Transport](https://mlsanity.com/paper/1803.00567/) (2018)
- [Reinforcement Learning with Deep Energy-Based Policies](https://mlsanity.com/paper/1702.08165/) (2017)
- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [Stochastic Gradient Hamiltonian Monte Carlo](https://mlsanity.com/paper/1402.4102/) (2014)
- [Black Box Variational Inference](https://mlsanity.com/paper/1401.0118/) (2013)
- [A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluation](https://mlsanity.com/paper/1602.03253/) (2016)
- [Fast Computation of Wasserstein Barycenters](https://mlsanity.com/paper/1310.4375/) (2013)
- [A Conceptual Introduction to Hamiltonian Monte Carlo](https://mlsanity.com/paper/1701.02434/) (2017)
- [Stein Variational Gradient Descent as Gradient Flow](https://mlsanity.com/paper/1704.07520/) (2017)
- [Maximum Mean Discrepancy Gradient Flow](https://mlsanity.com/paper/1906.04370/) (2019)
- [A Variational Perspective on Accelerated Methods in Optimization](https://mlsanity.com/paper/1603.04245/) (2016)
- [Deep Bayesian Active Learning for Natural Language Processing: Results of a Large-Scale Empirical Study](https://mlsanity.com/paper/1808.05697/) (2018)
- [A Non-Asymptotic Analysis for Stein Variational Gradient Descent](https://mlsanity.com/paper/2006.09797/) (2020)
- [Kernel Stein Discrepancy Descent](https://mlsanity.com/paper/2105.09994/) (2021)
- [What Can Be Transferred: Unsupervised Domain Adaptation for Endoscopic Lesions Segmentation](https://mlsanity.com/paper/2004.11500/) (2020)
- [Accelerating Langevin Sampling with Birth-death](https://mlsanity.com/paper/1905.09863/) (2019)
- [Stein Variational Policy Gradient](https://mlsanity.com/paper/1704.02399/) (2017)
- [A Unified Particle-Optimization Framework for Scalable Bayesian Sampling](https://mlsanity.com/paper/1805.11659/) (2018)
- [Stein Points](https://mlsanity.com/paper/1803.10161/) (2018)
- [Accelerated Information Gradient flow](https://mlsanity.com/paper/1909.02102/) (2019)
- [Optimal Transport for Treatment Effect Estimation](https://mlsanity.com/paper/2310.18286/) (2023)
- [Global convergence of neuron birth-death dynamics](https://mlsanity.com/paper/1902.01843/) (2019)
- [Learning to Draw Samples: With Application to Amortized MLE for Generative Adversarial Learning](https://mlsanity.com/paper/1611.01722/) (2016)
- [Stochastic Particle-Optimization Sampling and the Non-Asymptotic Convergence Theory](https://mlsanity.com/paper/1809.01293/) (2018)
- [Sampling with Mollified Interaction Energy Descent](https://mlsanity.com/paper/2210.13400/) (2022)
- [Accelerated Flow for Probability Distributions](https://mlsanity.com/paper/1901.03317/) (2019)
- [Stein Variational Gradient Descent: many-particle and long-time asymptotics](https://mlsanity.com/paper/2102.12956/) (2021)

## Built on by (most cited first, 9 of 9)

- [BELM: Bidirectional Explicit Linear Multi-step Sampler for Exact Inversion in Diffusion Models](https://mlsanity.com/paper/2410.07273/) (2024)
- [Mass Concept Erasure in Diffusion Models with Concept Hierarchy](https://mlsanity.com/paper/2601.03305/) (2026)
- [DriveDiTFit: Fine-tuning Diffusion Transformers for Autonomous Driving](https://mlsanity.com/paper/2407.15661/) (2024)
- [FG-OrIU: Towards Better Forgetting via Feature-Gradient Orthogonality for Incremental Unlearning](https://mlsanity.com/paper/2601.13578/) (2026)
- [Efficiently Access Diffusion Fisher: Within the Outer Product Span Space](https://mlsanity.com/paper/2505.23264/) (2025)
- [Unleashing High-Quality Image Generation in Diffusion Sampling Using Second-Order Levenberg-Marquardt-Langevin](https://mlsanity.com/paper/2505.24222/) (2025)
- [Rethinking the Diffusion Models for Numerical Tabular Data Imputation from the Perspective of Wasserstein Gradient Flow](https://mlsanity.com/paper/2406.15762/) (2024)
- [Rethinking the Flow-Based Gradual Domain Adaptation: A Semi-Dual Optimal Transport Perspective](https://mlsanity.com/paper/2602.01179/) (2026)
- [Slack More, Predict Better: Proximal Relaxation for Probabilistic Latent Variable Model-based Soft Sensors](https://mlsanity.com/paper/2603.11473/) (2026)

## Abstract (arXiv)

Particle-based Variational Inference (ParVI) methods approximate the target distribution by iteratively evolving finite weighted particle systems. Recent advances of ParVI methods reveal the benefits of accelerated position update strategies and dynamic weight adjustment approaches. In this paper, we propose the first ParVI framework that possesses both accelerated position update and dynamical weight adjustment simultaneously, named the General Accelerated Dynamic-Weight Particle-based Variational Inference (GAD-PVI) framework. Generally, GAD-PVI simulates the semi-Hamiltonian gradient flow on a novel Information-Fisher-Rao space, which yields an additional decrease on the local functional dissipation. GAD-PVI is compatible with different dissimilarity functionals and associated smoothing approaches under three information metrics. Experiments on both synthetic and real-world data demonstrate the faster convergence and reduced approximation error of GAD-PVI methods over the state-of-the-art.