mlsanityData to 7 Oct 2026

Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm

arXiv 1608.04471 ↗First posted 16 Aug 2016stat.ML cs.LGcited by 498

Not on the map. Why?

Cited by 31 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We propose a general purpose variational inference algorithm that forms a natural counterpart of gradient descent for optimization. Our method iteratively transports a set of particles to match the target distribution, by applying a form of functional gradient descent that minimizes the KL divergence. Empirical studies are performed on various real world models and datasets, on which our method is competitive with existing state-of-the-art methods. The derivation of our method is based on a new theoretical result that connects the derivative of KL divergence under smooth transforms with Stein's identity and a recently proposed kernelized Stein discrepancy, which is of independent interest.

Key facts. Cited by 498 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Aug 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 83; 2022: 82; 2023: 78; 2024: 99; 2025: 79; 2026: 76.

498 papers.

Sort:

By area:

  1. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  2. 2021Transformers Can Do Bayesian Inferencecited by 338
  3. 2023A Variational Perspective on Solving Inverse Problems with Diffusion Modelscited by 217
  4. 2023Towards Building the Federated GPT: Federated Instruction Tuningcited by 184
  5. 2021Path Integral Sampler: a stochastic control approach for samplingcited by 175
  6. 2021DiBS: Differentiable Bayesian Structure Learningcited by 105
  7. 2021Repulsive Deep Ensembles are Bayesiancited by 103
  8. 2022GFlowNets and variational inferencecited by 103
  9. 2021Priors in Bayesian Deep Learning: A Reviewcited by 101
  10. 2021Oops I Took A Gradient: Scalable Sampling for Discrete Distributionscited by 92
  11. 2021Annealed Flow Transport Monte Carlocited by 90
  12. 2022Variational inference via Wasserstein gradient flowscited by 84
38 more
  1. 2021Large-Scale Wasserstein Gradient Flowscited by 76
  2. 2020Safe Imitation Learning via Fast Bayesian Reward Inference from Preferencescited by 73
  3. 2023Particle Guidance: non-I.I.D. Diverse Sampling with Diffusion Modelscited by 65
  4. 2024Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AIcited by 60
  5. 2021Kernel Stein Discrepancy Descentcited by 57
  6. 2022Diversify and Disambiguate: Learning From Underspecified Datacited by 55
  7. 2022Equivariant Descriptor Fields: SE(3)-Equivariant Energy-Based Models for End-to-End Visual Robotic Manipulation Learningcited by 54
  8. 2021Optimizing Functionals on the Space of Probabilities with Input Convex Neural Networkscited by 53
  9. 2021Variational Wasserstein gradient flowcited by 53
  10. 2022Sequential Neural Score Estimation: Likelihood-Free Inference with Conditional Score Based Diffusion Modelscited by 50
  11. 2023Joint Bayesian Inference of Graphical Structure and Parameters with a Single Generative Flow Networkcited by 50
  12. 2022Interventions, Where and How? Experimental Design for Causal Models at Scalecited by 49
  13. 2022Probability flow solution of the Fokker-Planck equationcited by 46
  14. 2021Diminishing Uncertainty within the Training Pool: Active Learning for Medical Image Segmentationcited by 44
  15. 2023Reverse Diffusion Monte Carlocited by 41
  16. 2021LiBRe: A Practical Bayesian Approach to Adversarial Detectioncited by 40
  17. 2022Normalizing flow neural networks by JKO schemecited by 39
  18. 2022Visual Attention Methods in Deep Learning: An In-Depth Surveycited by 37
  19. 2023Forward-backward Gaussian variational inference via JKO in the Bures-Wasserstein Spacecited by 36
  20. 2023Generative Sliced MMD Flows with Riesz Kernelscited by 36
  21. 2022Active Bayesian Causal Inferencecited by 34
  22. 2023BayesDAG: Gradient-Based Posterior Inference for Causal Discoverycited by 33
  23. 2021Sketching Curvature for Efficient Out-of-Distribution Detection for Deep Neural Networkscited by 32
  24. 2022Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reductioncited by 32
  25. 2023A Rigorous Link between Deep Ensembles and (Variational) Bayesian Methodscited by 31
  26. 2022Birth-death dynamics for sampling: Global convergence, approximations and their asymptoticscited by 30
  27. 2023BayesDiff: Estimating Pixel-wise Uncertainty in Diffusion via Bayesian Inferencecited by 30
  28. 2022Improving Task-free Continual Learning by Distributionally Robust Memory Evolutioncited by 29
  29. 2023Improved dimension dependence of a proximal algorithm for samplingcited by 29
  30. 2023A mean-field games laboratory for generative modelingcited by 28
  31. 2022Accelerated Linearized Laplace Approximation for Bayesian Deep Learningcited by 27
  32. 2023DynGFN: Towards Bayesian Inference of Gene Regulatory Networks with GFlowNetscited by 27
  33. 2024Sampling in Unit Time with Kernel Fisher-Rao Flowcited by 27
  34. 2024Score Distillation Sampling with Learned Manifold Correctivecited by 27
  35. 2022Particle algorithms for maximum likelihood training of latent variable modelscited by 26
  36. 2022BaCaDI: Bayesian Causal Discovery with Unknown Interventionscited by 26
  37. 2021Sqrt(d) Dimension Dependence of Langevin Monte Carlocited by 25
  38. 2021Variational Automatic Curriculum Learning for Sparse-Reward Cooperative Multi-Agent Problemscited by 25

50 most cited of 498 All 498 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2021Variational Rejection Particle Filtering
  2. 2022A Framework for Improving the Reliability of Black-box Variational Inference
  3. 2025Theory and computation for structured variational inference
  4. 2025A Frequentist Statistical Introduction to Variational Inference, Autoencoders, and Diffusion Models
  5. 2022Amortized backward variational inference in nonlinear state-space models
  6. 2023Transport meets Variational Inference: Controlled Monte Carlo Diffusions
  7. 2023Hierarchical Semi-Implicit Variational Inference with Application to Diffusion Model Acceleration
  8. 2024Denoising Diffusion Variational Inference: Diffusion Models as Expressive Variational Posteriors