mlsanityData to 7 Oct 2026

Convergence of Stein Variational Gradient Descent under a Weaker Smoothness Condition

arXiv 2206.00508 ↗First posted 1 Jun 2022math.ST cs.LG math.PRcited by 16

Not on the map. Why?

Abstract (arXiv)

Stein Variational Gradient Descent (SVGD) is an important alternative to the Langevin-type algorithms for sampling from probability distributions of the form π(x) propto exp(-V(x)). In the existing theory of Langevin-type algorithms and SVGD, the potential function V is often assumed to be L-smooth. However, this restrictive condition excludes a large class of potential functions such as polynomials of degree greater than 2. Our paper studies the convergence of the SVGD algorithm for distributions with (L0,L1)-smooth potentials. This relaxed smoothness assumption was introduced by Zhang et al. [2019a] for the analysis of gradient clipping algorithms. With the help of trajectory-independent auxiliary conditions, we provide a descent lemma establishing that the algorithm decreases the KL divergence at each iteration and prove a complexity bound for SVGD in the population limit in terms of the Stein Fisher information.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 46 of its 87 references that are in the map. First posted on arXiv 1 Jun 2022.

Builds on

46 of 87 references.

References · 46

  1. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 498
  2. 2019Why gradient clipping accelerates training: A theoretical justification for adaptivitycited by 335
  3. 2017Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysiscited by 242
  4. 2016A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluationcited by 236
  5. 2019Rapid Convergence of the Unadjusted Langevin Algorithm: Isoperimetry Sufficescited by 224
  6. 2014Theoretical guarantees for approximate sampling from smooth and log-concave densitiescited by 206
  7. 2015Non-asymptotic convergence analysis for the Unadjusted Langevin Algorithmcited by 195
  8. 2017Stein Variational Gradient Descent as Gradient Flowcited by 149
  9. 2016High-dimensional Bayesian inference via the Unadjusted Langevin Algorithmcited by 144
  10. 2018Analysis of Langevin Monte Carlo via convex optimizationcited by 126
  11. 2018Sampling as optimization in the space of measures: The Langevin dynamics as a composite optimization problemcited by 123
  12. 2017Measuring Sample Quality with Kernelscited by 119
34 more
  1. 2018Log-concave sampling: Metropolis-Hastings algorithms are fastcited by 117
  2. 2017User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradientcited by 116
  3. 2017Convergence of Langevin MCMC in KL-divergencecited by 86
  4. 2016Efficient Bayesian computation by proximal Markov chain Monte Carlo: when Langevin meets Moreaucited by 83
  5. 2019On the geometry of Stein variational gradient descentcited by 82
  6. 2019The Randomized Midpoint Method for Log-Concave Samplingcited by 77
  7. 2014A statistical model for tensor PCAcited by 72
  8. 2018Sharp convergence rates for Langevin dynamics in the nonconvex settingcited by 67
  9. 2022Towards a Theory of Non-Log-Concave Sampling: First-Order Stationarity Guarantees for Langevin Monte Carlocited by 63
  10. 2020A Non-Asymptotic Analysis for Stein Variational Gradient Descentcited by 61
  11. 2020On the Convergence of Langevin Monte Carlo: The Interplay between Tail Growth and Smoothnesscited by 55
  12. 2020Efficient constrained sampling via the mirror-Langevin algorithmcited by 55
  13. 2018A Stein variational Newton methodcited by 49
  14. 2017Stein Variational Policy Gradientcited by 42
  15. 2020Exponential ergodicity of mirror-Langevin diffusionscited by 37
  16. 2018Scaling limit of the Stein variational gradient descent: the mean field regimecited by 35
  17. 2020Federated Generalized Bayesian Learning via Distributed Stein Variational Gradient Descentcited by 34
  18. 2019Stein Variational Gradient Descent With Matrix-Valued Kernelscited by 32
  19. 2017The Tamed Unadjusted Langevin Algorithmcited by 31
  20. 2019Proximal Langevin Algorithm: Rapid Convergence Under Isoperimetrycited by 31
  21. 2020Stochastic Stein Discrepanciescited by 30
  22. 2019Langevin Monte Carlo without smoothnesscited by 27
  23. 2021The Langevin Monte Carlo algorithm in the non-smooth log-concave casecited by 26
  24. 2018On the Theory of Variance Reduction for Stochastic Gradient Monte Carlocited by 19
  25. 2019Is There an Analog of Nesterov Acceleration for MCMC?cited by 19
  26. 2021Sampling with Mirrored Stein Operatorscited by 17
  27. 2019Bounding the error of discretized Langevin algorithms for non-strongly log-concave targetscited by 16
  28. 2018Stein Variational Gradient Descent as Moment Matchingcited by 15
  29. 2021Stein Variational Gradient Descent: many-particle and long-time asymptoticscited by 15
  30. 2017Fractional Langevin Monte Carlo: Exploring L'evy Driven Stochastic Differential Equations for Markov Chain Monte Carlocited by 12
  31. 2019A stochastic version of Stein Variational Gradient Descent for efficient samplingcited by 10
  32. 2019Scalable Thompson Sampling via Optimal Transportcited by 7
  33. 2017Learning Structural Weight Uncertainty for Sequential Decision-Makingcited by 5
  34. 2020Penalized Langevin dynamics with vanishing penalty for smooth and log-concave targetscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 5; 2024: 5; 2025: 0; 2026: 3.

16 papers.

Sort:

  1. 2022A Finite-Particle Convergence Rate for Stein Variational Gradient Descentcited by 22
  2. 2023Variance-reduced Clipping for Non-convex Optimizationcited by 22
  3. 2023Towards Understanding the Dynamics of Gaussian-Stein Variational Gradient Descentcited by 13
  4. 2024Improved Finite-Particle Convergence Rates for Stein Variational Gradient Descentcited by 13
  5. 2023Provably Fast Finite Particle Variants of SVGD via Virtual Particle Stochastic Approximationcited by 6
  6. 2023Towards a Better Theoretical Understanding of Independent Subnetwork Trainingcited by 6
  7. 2022A Note on the Convergence of Mirrored Stein Variational Gradient Descent under (L0,L1)-Smoothness Conditioncited by 4
  8. 2023Learning Rate Free Sampling in Constrained Domainscited by 3
  9. 2024Bayesian Multi-Task Transfer Learning for Soft Prompt Tuningcited by 3
  10. 2026Finite-Particle Rates for Regularized Stein Variational Gradient Descentcited by 3
  11. 2022Improved Stein Variational Gradient Descent with Importance Weightscited by 2
  12. 2024Stein Boltzmann Sampling: A Variational Approach for Global Optimizationcited by 2
4 more
  1. 2024Long-time asymptotics of noisy SVGD outside the population limitcited by 2
  2. 2024A Stochastic Quasi-Newton Method for Non-convex Optimization with Non-uniform Smoothnesscited by 1
  3. 2026To discretize continually: Mean shift interacting particle systems for Bayesian inferencecited by 1
  4. 2026Complexity of Non-Log-Concave Sampling in Fisher Informationcited by 0