mlsanityData to 7 Oct 2026

Annealed Stein Variational Gradient Descent

arXiv 2101.09815 ↗First posted 24 Jan 2021cs.LGcited by 24

Not on the map. Why?

Abstract (arXiv)

Particle based optimization algorithms have recently been developed as sampling methods that iteratively update a set of particles to approximate a target distribution. In particular Stein variational gradient descent has gained attention in the approximate inference literature for its flexibility and accuracy. We empirically explore the ability of this method to sample from multi-modal distributions and focus on two important issues: (i) the inability of the particles to escape from local modes and (ii) the inefficacy in reproducing the density of the different regions. We propose an annealing schedule to solve these issues and show, through various experiments, how this simple solution leads to significant improvements in mode coverage, without invalidating any theoretical properties of the original algorithm.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 12 of its 21 references that are in the map. First posted on arXiv 24 Jan 2021.

Builds on

12 of 21 references.

References · 12

  1. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  2. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 498
  3. 2017Snapshot Ensembles: Train 1, get M for freecited by 349
  4. 2019Cyclical Stochastic Gradient MCMC for Bayesian Deep Learningcited by 203
  5. 2019Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishingcited by 191
  6. 2019On the geometry of Stein variational gradient descentcited by 82
  7. 2017Message Passing Stein Variational Gradient Descentcited by 47
  8. 2018Improving Explorability in Variational Inference with Annealed Variational Objectivescited by 24
  9. 2014Variational Temperingcited by 20
  10. 2018Stochastic Particle-Optimization Sampling and the Non-Asymptotic Convergence Theorycited by 20
  11. 2018Stein Variational Gradient Descent Without Gradientcited by 12
  12. 2020Kernel Stein Generative Modelingcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 3; 2023: 4; 2024: 7; 2025: 3; 2026: 3.

24 papers.

Sort:

By area:

  1. 2021Repulsive Deep Ensembles are Bayesiancited by 103
  2. 2023Particle Guidance: non-I.I.D. Diverse Sampling with Diffusion Modelscited by 65
  3. 2021On Stein Variational Neural Network Ensemblescited by 21
  4. 2021Neural Variational Gradient Descentcited by 19
  5. 2024Repulsive Latent Score Distillation for Solving Inverse Problemscited by 15
  6. 2022Towards Healing the Blindness of Score Matchingcited by 13
  7. 2021Learning Equivariant Energy Based Models with Equivariant Stein Variational Gradient Descentcited by 10
  8. 2023Constrained Stein Variational Trajectory Optimizationcited by 8
  9. 2023Adaptive Annealed Importance Sampling with Constant Rate Progresscited by 7
  10. 2022Stein Variational Goal Generation for adaptive Exploration in Multi-Goal Reinforcement Learningcited by 5
  11. 2022A stochastic Stein Variational Newton methodcited by 4
  12. 2024Annealed Stein Variational Gradient Descent for Improved Uncertainty Estimation in Full-Waveform Inversioncited by 4
12 more
  1. 2024Long-time asymptotics of noisy SVGD outside the population limitcited by 2
  2. 2024Stein Variational Evolution Strategiescited by 2
  3. 2023Long-tailed Classification from a Bayesian-decision-theory Perspectivecited by 1
  4. 2024ELBOing Stein: Variational Bayes with Stein Mixture Inferencecited by 1
  5. 2025Making Reliable and Flexible Decisions in Long-tailed Classificationcited by 1
  6. 2024Accelerating Convergence of Stein Variational Gradient Descent via Deep Unfoldingcited by 0
  7. 2024Stein Variational Newton Neural Network Ensemblescited by 0
  8. 2025Entropy-regularized Gradient Estimators for Approximate Bayesian Inferencecited by 0
  9. 2025A Framework for Controllable Multi-objective Learning with Annealed Stein Variational Hypernetworkscited by 0
  10. 2026Towards Intrinsically Calibrated Uncertainty Quantification in Industrial Data-Driven Models via Diffusion Samplercited by 0
  11. 2026Variational Inference via Entropic Transport Descentcited by 0
  12. 2026Wasserstein Gradient Flows and Forward-Only Diffusion Are Not Enough for Multimodal Samplingcited by 0