mlsanityData to 7 Oct 2026

Markov Chain Score Ascent: A Unifying Framework of Variational Inference with Markovian Gradients

arXiv 2206.06295 ↗First posted 13 Jun 2022cs.LG cs.AI stat.MLcited by 6

Not on the map. Why?

Abstract (arXiv)

Minimizing the inclusive Kullback-Leibler (KL) divergence with stochastic gradient descent (SGD) is challenging since its gradient is defined as an integral over the posterior. Recently, multiple methods have been proposed to run SGD with biased gradient estimates obtained from a Markov chain. This paper provides the first non-asymptotic convergence analysis of these methods by establishing their mixing rate and gradient variance. To do this, we demonstrate that these methods-which we collectively refer to as Markov chain score ascent (MCSA) methods-can be cast as special cases of the Markov chain gradient descent framework. Furthermore, by leveraging this new understanding, we develop a novel MCSA scheme, parallel MCSA (pMCSA), that achieves a tighter bound on the gradient variance. We demonstrate that this improved theoretical result translates to superior empirical performance.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 37 of its 103 references that are in the map. First posted on arXiv 13 Jun 2022.

Builds on

37 of 103 references.

References · 37

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016Variational Inference: A Review for Statisticianscited by 1,466
  3. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
  4. 2013Black Box Variational Inferencecited by 317
  5. 2015Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networkscited by 295
  6. 2016Automatic Differentiation Variational Inferencecited by 159
  7. 2016R'enyi Divergence Variational Inferencecited by 104
  8. 2017Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inferencecited by 97
  9. 2018SGD and Hogwild! Convergence Without the Bounded Gradients Assumptioncited by 94
  10. 2014Reweighted Wake-Sleepcited by 66
  11. 2018On Markov Chain Gradient Descentcited by 63
  12. 2018Importance Weighting and Variational Inferencecited by 55
25 more
  1. 2020Markovian Score Climbing: Variational Inference with KL(p||q)cited by 46
  2. 2019Non-asymptotic Analysis of Biased Stochastic Approximation Schemecited by 45
  3. 2021MCMC Variational Inference via Uncorrected Hamiltonian Annealingcited by 39
  4. 2015Black-box α-divergence Minimizationcited by 37
  5. 2021Differentiable Annealed Importance Sampling and the Perils of Gradient Noisecited by 36
  6. 2021Challenges and Opportunities in High-dimensional Variational Inferencecited by 35
  7. 2021Variational Refinement for Importance Sampling Using the Forward Kullback-Leibler Divergencecited by 29
  8. 2020f-Divergence Variational Inferencecited by 28
  9. 2018Variational Inference with Tail-adaptive f-Divergencecited by 25
  10. 2017Approximate Inference with Amortised MCMCcited by 22
  11. 2022Wide Mean-Field Bayesian Neural Networks Ignore the Datacited by 22
  12. 2018Overpruning in Variational Bayesian Neural Networkscited by 21
  13. 2020On the Difficulty of Unbiased Alpha Divergence Minimizationcited by 21
  14. 2019A Contrastive Divergence for Combining Variational Inference and MCMCcited by 20
  15. 2017Asymptotic Bias of Stochastic Gradient Searchcited by 19
  16. 2020Finite-Time Analysis of Stochastic Gradient Descent under Markov Randomnesscited by 18
  17. 2020Convergence Rates of Accelerated Markov Gradient Descent with Applications in Reinforcement Learningcited by 15
  18. 2020Non-asymptotic Convergence of Adam-type Reinforcement Learning Algorithms under Markovian Samplingcited by 15
  19. 2018Alpha-Beta Divergence For Variational Inferencecited by 12
  20. 2020Joint Stochastic Approximation and Its Application to Learning Discrete Latent Variable Modelscited by 10
  21. 2021Empirical Evaluation of Biased Methods for Alpha Divergence Minimizationcited by 10
  22. 2017Perturbative Black Box Variational Inferencecited by 8
  23. 2022Transport Score Climbing: Variational Inference Using Forward KL and Adaptive Neural Transportcited by 8
  24. 2018Metropolis-Hastings view on variational inference and adversarial trainingcited by 5
  25. 2019A Rule for Gradient Estimator Selection, with an Application to Variational Inferencecited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 4; 2025: 0; 2026: 1.

6 papers.

  1. 2023On the Convergence of Black-Box Variational Inferencecited by 22
  2. 2024Disentangling impact of capacity, objective, batchsize, estimators, and step-size on flow VIcited by 5
  3. 2024Differentiable Annealed Importance Sampling Minimizes The Symmetrized Kullback-Leibler Divergence Between Initial and Target Distributioncited by 2
  4. 2024Understanding and mitigating difficulties in posterior predictive evaluationcited by 1
  5. 2026A hitchhiker's guide to Poisson gradient estimationcited by 1
  6. 2024MCMC-driven learningcited by 0