mlsanityData to 14 Oct 2026

Particle-based Variational Inference with Preconditioned Functional Gradient Flow

arXiv 2211.13954 ↗First posted 25 Nov 2022stat.ML cs.LGcited by 20

Not on the map. Why?

Abstract (arXiv)

Particle-based variational inference (VI) minimizes the KL divergence between model samples and the target posterior with gradient flow estimates. With the popularity of Stein variational gradient descent (SVGD), the focus of particle-based VI algorithms has been on the properties of functions in Reproducing Kernel Hilbert Space (RKHS) to approximate the gradient flow. However, the requirement of RKHS restricts the function class and algorithmic flexibility. This paper offers a general solution to this problem by introducing a functional regularization term that encompasses the RKHS norm as a special case. This allows us to propose a new particle-based VI algorithm called preconditioned functional gradient flow (PFG). Compared to SVGD, PFG has several advantages. It has a larger function class, improved scalability in large particle-size scenarios, better adaptation to ill-conditioned distributions, and provable continuous-time convergence in KL divergence. Additionally, non-linear function classes such as neural networks can be incorporated to estimate the gradient flow. Our theory and experiments demonstrate the effectiveness of the proposed framework.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Builds on 32 of its 62 references that are in the map. First posted on arXiv 25 Nov 2022.

Builds on

32 of 62 references.

References · 32

  1. 2016Variational Inference: A Review for Statisticianscited by 1,472
  2. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 507
  3. 2014Stochastic Gradient Hamiltonian Monte Carlocited by 343
  4. 2013Black Box Variational Inferencecited by 319
  5. 2016A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluationcited by 236
  6. 2019Rapid Convergence of the Unadjusted Langevin Algorithm: Isoperimetry Sufficescited by 227
  7. 2017Stein Variational Gradient Descent as Gradient Flowcited by 152
  8. 2016A Kernel Test of Goodness of Fitcited by 143
  9. 2018Sampling as optimization in the space of measures: The Langevin dynamics as a composite optimization problemcited by 123
  10. 2015Preconditioned Stochastic Gradient Langevin Dynamics for Deep Neural Networkscited by 121
  11. 2019On the geometry of Stein variational gradient descentcited by 82
  12. 2021Large-Scale Wasserstein Gradient Flowscited by 77
20 more
  1. 2020SVGD as a kernelized Wasserstein gradient flow of the chi-squared divergencecited by 64
  2. 2021Optimizing Functionals on the Space of Probabilities with Input Convex Neural Networkscited by 53
  3. 2021Variational Wasserstein gradient flowcited by 53
  4. 2018A Stein variational Newton methodcited by 50
  5. 2017Message Passing Stein Variational Gradient Descentcited by 47
  6. 2015Incremental Gradient, Subgradient, and Proximal Methods for Convex Optimization: A Surveycited by 45
  7. 2019Accelerated Information Gradient flowcited by 39
  8. 2019Stein Variational Gradient Descent With Matrix-Valued Kernelscited by 32
  9. 2017Learning to Draw Samples with Amortized Stein Variational Gradient Descentcited by 28
  10. 2020Information Newton's flow: second-order optimization method in probability spacecited by 24
  11. 2021Projected Wasserstein gradient descent for high-dimensional Bayesian inferencecited by 24
  12. 2020Sliced Kernelized Stein Discrepancycited by 23
  13. 2016Learning to Draw Samples: With Application to Amortized MLE for Generative Adversarial Learningcited by 21
  14. 2019Projected Stein Variational Newton: A Fast and Scalable Bayesian Inference Method in High Dimensionscited by 21
  15. 2018Stein Neural Samplercited by 19
  16. 2021Neural Variational Gradient Descentcited by 19
  17. 2021Wasserstein Proximal of GANscited by 15
  18. 2022Grassmann Stein Variational Gradient Descentcited by 12
  19. 2022Optimal Neural Network Approximation of Wasserstein Gradient Direction via Convex Optimizationcited by 9
  20. 2018Gradient Layer: Enhancing the Convergence of Adversarial Training for Generative Modelscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 6; 2024: 11; 2025: 0; 2026: 3.

20 papers.

Sort:

By area:

  1. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,219
  2. 2023Reverse Diffusion Monte Carlocited by 41
  3. 2023Generative Sliced MMD Flows with Riesz Kernelscited by 37
  4. 2024Retrieval-Augmented Score Distillation for Text-to-3D Generationcited by 14
  5. 2023Particle-based Variational Inference with Generalized Wasserstein Gradient Flowcited by 10
  6. 2023Neural Wasserstein Gradient Flows for Maximum Mean Discrepancies with Riesz Kernelscited by 9
  7. 2024Kernel Semi-Implicit Variational Inferencecited by 8
  8. 2024Path-Guided Particle-based Samplingcited by 8
  9. 2024Kernel Approximation of Fisher-Rao Gradient Flowscited by 7
  10. 2023Wasserstein Gradient Flow over Variational Parameter Space for Variational Inferencecited by 5
  11. 2024Faster Sampling via Stochastic Gradient Proximal Samplercited by 5
  12. 2024GradeADreamer: Enhanced Text-to-3D Generation Using Gaussian Splatting and Multi-View Diffusioncited by 4
8 more
  1. 2024Rethinking the Diffusion Models for Numerical Tabular Data Imputation from the Perspective of Wasserstein Gradient Flowcited by 3
  2. 2024Perturb-and-Revise: Flexible 3D Editing with Generative Trajectoriescited by 3
  3. 2024Semi-Implicit Functional Gradient Flow for Efficient Samplingcited by 1
  4. 2024ELBOing Stein: Variational Bayes with Stein Mixture Inferencecited by 1
  5. 2026A Gradient Flow Perspective on Minimum MMD Estimationcited by 1
  6. 2024Functional Gradient Flows for Constrained Samplingcited by 0
  7. 2026Analyzing and Improving Diffusion Models for Time-Series Data Imputation: A Proximal Recursion Perspectivecited by 0
  8. 2026Slack More, Predict Better: Proximal Relaxation for Probabilistic Latent Variable Model-based Soft Sensorscited by 0