mlsanityData to 14 Oct 2026

Neural Variational Gradient Descent

arXiv 2107.10731 ↗First posted 22 Jul 2021cs.LG stat.CO stat.MLcited by 19

Not on the map. Why?

Abstract (arXiv)

Particle-based approximate Bayesian inference approaches such as Stein Variational Gradient Descent (SVGD) combine the flexibility and convergence guarantees of sampling methods with the computational benefits of variational inference. In practice, SVGD relies on the choice of an appropriate kernel function, which impacts its ability to model the target distribution -- a challenging problem with only heuristic solutions. We propose Neural Variational Gradient Descent (NVGD), which is based on parameterizing the witness function of the Stein discrepancy by a deep neural network whose parameters are learned in parallel to the inference, mitigating the necessity to make any kernel choices whatsoever. We empirically evaluate our method on popular synthetic inference problems, real-world Bayesian linear regression, and Bayesian neural network inference.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 28 of its 48 references that are in the map. First posted on arXiv 22 Jul 2021.

Builds on

28 of 48 references.

References · 28

  1. 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 594
  2. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 507
  3. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 308
  4. 2019Rapid Convergence of the Unadjusted Langevin Algorithm: Isoperimetry Sufficescited by 227
  5. 2017Stein Variational Gradient Descent as Gradient Flowcited by 152
  6. 2021Bayesian Neural Network Priors Revisitedcited by 121
  7. 2021Scalable Marginal Likelihood Estimation for Model Selection in Deep Learningcited by 106
  8. 2021Repulsive Deep Ensembles are Bayesiancited by 104
  9. 2021Priors in Bayesian Deep Learning: A Reviewcited by 101
  10. 2015Measuring Sample Quality with Stein's Methodcited by 94
  11. 2020SVGD as a kernelized Wasserstein gradient flow of the chi-squared divergencecited by 64
  12. 2018A Stein variational Newton methodcited by 50
16 more
  1. 2017Message Passing Stein Variational Gradient Descentcited by 47
  2. 2019Function Space Particle Optimization for Bayesian Neural Networkscited by 41
  3. 2020Projected Stein Variational Gradient Descentcited by 33
  4. 2020Stochastic Stein Discrepanciescited by 30
  5. 2021Exact Langevin Dynamics with Stochastic Gradientscited by 30
  6. 2020Stacking for Non-mixing Bayesian Computations: The Curse and Blessing of Multimodal Posteriorscited by 29
  7. 2021Annealed Stein Variational Gradient Descentcited by 26
  8. 2020Sliced Kernelized Stein Discrepancycited by 23
  9. 2021On Stein Variational Neural Network Ensemblescited by 21
  10. 2021BNNpriors: A library for Bayesian neural network inference with different prior distributionscited by 20
  11. 2018Stein Neural Samplercited by 19
  12. 2018Stochastic Gradient MCMC with Repulsive Forcescited by 18
  13. 2016Operator Variational Inferencecited by 16
  14. 2020Stein Self-Repulsive Dynamics: Benefits From Past Samplescited by 6
  15. 2020Kernel Stein Generative Modelingcited by 5
  16. 2019Applying SVGD to Bayesian Neural Networks for Cyclical Time-Series Prediction and Inferencecited by 3

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 6; 2024: 9; 2025: 1.

19 papers.

Sort:

By area:

  1. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,219
  2. 2023Generative Sliced MMD Flows with Riesz Kernelscited by 37
  3. 2024Training Neural Samplers with Reverse Diffusive KL Divergencecited by 21
  4. 2022Particle-based Variational Inference with Preconditioned Functional Gradient Flowcited by 20
  5. 2023Self-Consistent Velocity Matching of Probability Flowscited by 17
  6. 2023Entropy-based Training Methods for Scalable Neural Implicit Samplercited by 13
  7. 2023Particle-based Variational Inference with Generalized Wasserstein Gradient Flowcited by 10
  8. 2022Optimal Neural Network Approximation of Wasserstein Gradient Direction via Convex Optimizationcited by 9
  9. 2023Neural Wasserstein Gradient Flows for Maximum Mean Discrepancies with Riesz Kernelscited by 9
  10. 2022Are you using test log-likelihood correctly?cited by 8
  11. 2024Path-Guided Particle-based Samplingcited by 8
  12. 2024Energy based diffusion generator for efficient sampling of Boltzmann distributionscited by 7
7 more
  1. 2024Stein Variational Evolution Strategiescited by 2
  2. 2024Bayesian Deep Learning for Remaining Useful Life Estimation via Stein Variational Gradient Descentcited by 1
  3. 2024Semi-Implicit Functional Gradient Flow for Efficient Samplingcited by 1
  4. 2024Accelerating Convergence of Stein Variational Gradient Descent via Deep Unfoldingcited by 0
  5. 2024Functional Gradient Flows for Constrained Samplingcited by 0
  6. 2024Denoising Fisher Training For Neural Implicit Samplerscited by 0
  7. 2025Adaptive Kernel Selection for Stein Variational Gradient Descentcited by 0