mlsanityData to 7 Oct 2026

Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inference

arXiv 1703.09194 ↗First posted 27 Mar 2017stat.ML cs.LGcited by 97

Not on the map. Why?

Abstract (arXiv)

We propose a simple and general variant of the standard reparameterized gradient estimator for the variational evidence lower bound. Specifically, we remove a part of the total derivative with respect to the variational parameters that corresponds to the score function. Removing this term produces an unbiased gradient estimator whose variance approaches zero as the approximate posterior approaches the exact posterior. We analyze the behavior of this gradient estimator theoretically and empirically, and generalize it to more complex variational distributions such as mixtures and importance-weighted posteriors.

Key facts. Cited by 97 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Mar 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 0; 2021: 13; 2022: 16; 2023: 25; 2024: 23; 2025: 10; 2026: 9.

97 papers.

Sort:

By area:

  1. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  2. 2019Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Modelscited by 177
  3. 2023Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Modelscited by 169
  4. 2022An optimal control perspective on diffusion-based generative modelingcited by 145
  5. 2023Improved sampling via learned diffusionscited by 103
  6. 2023Generative Time Series Forecasting with Diffusion, Denoise, and Disentanglementcited by 89
  7. 2024TSD-SR: One-Step Diffusion with Target Score Distillation for Real-World Image Super-Resolutioncited by 83
  8. 2021BCD Nets: Scalable Variational Approaches for Bayesian Causal Discoverycited by 68
  9. 2023Score Regularized Policy Optimization through Diffusion Behaviorcited by 67
  10. 2023Diffusion Generative Flow Samplers: Improving learning signals through partial trajectory optimizationcited by 57
  11. 2021Bayesian Learning via Neural Schr"odinger-F"ollmer Flowscited by 56
  12. 2023Training Chain-of-Thought via Latent-Variable Inferencecited by 46
38 more
  1. 2022Variational methods for simulation-based inferencecited by 43
  2. 2024Score Distillation via Reparametrized DDIMcited by 41
  3. 2023Stochastic Optimal Control Matchingcited by 39
  4. 2021Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equationscited by 38
  5. 2021Challenges and Opportunities in High-dimensional Variational Inferencecited by 35
  6. 2022Mitigating Modality Collapse in Multimodal VAEs via Impartial Optimizationcited by 32
  7. 2023Provable convergence guarantees for black-box variational inferencecited by 29
  8. 2023SteinDreamer: Variance Reduction for Text-to-3D Score Distillation via Stein Identitycited by 23
  9. 2022Gradients should stay on Path: Better Estimators of the Reverse- and Forward KL Divergence for Normalizing Flowscited by 22
  10. 2022A Unified Perspective on Natural Gradient Variational Inference with Gaussian Mixture Modelscited by 22
  11. 2023On the Convergence of Black-Box Variational Inferencecited by 22
  12. 2023Transport meets Variational Inference: Controlled Monte Carlo Diffusionscited by 21
  13. 2021Amortized Variational Inference for Simple Hierarchical Modelscited by 19
  14. 2021Generalized Doubly Reparameterized Gradient Estimatorscited by 14
  15. 2021Nested Variational Inferencecited by 14
  16. 2024Scalable Bayesian Learning with posteriorscited by 13
  17. 2022ProbNeRF: Uncertainty-Aware Inference of 3D Shapes from 2D Imagescited by 12
  18. 2023MonoFlow: Rethinking Divergence GANs via the Perspective of Wasserstein Gradient Flowscited by 12
  19. 2024A Taxonomy of Loss Functions for Stochastic Optimal Controlcited by 11
  20. 2021Empirical Evaluation of Biased Methods for Alpha Divergence Minimizationcited by 10
  21. 2022Path-Gradient Estimators for Continuous Normalizing Flowscited by 10
  22. 2023Linear Convergence of Black-Box Variational Inference: Should We Stick the Landing?cited by 10
  23. 2023Bridging the Gap Between Variational Inference and Wasserstein Gradient Flowscited by 8
  24. 2025Model Immunization from a Condition Number Perspectivecited by 8
  25. 2024Particle Semi-Implicit Variational Inferencecited by 7
  26. 2021A variational approximate posterior for the deep Wishart processcited by 6
  27. 2022Variational Sparse Coding with Learned Thresholdingcited by 6
  28. 2022Markov Chain Score Ascent: A Unifying Framework of Variational Inference with Markovian Gradientscited by 6
  29. 2022Cooperation in the Latent Space: The Benefits of Adding Mixture Components in Variational Autoencoderscited by 6
  30. 2023Practical and Matching Gradient Variance Bounds for Black-Box Variational Bayesian Inferencecited by 6
  31. 2023Sample Average Approximation for Black-Box VIcited by 6
  32. 2024Partially Stochastic Infinitely Deep Bayesian Neural Networkscited by 6
  33. 2024On Divergence Measures for Training GFlowNetscited by 6
  34. 2025Control Consistency Losses for Diffusion Bridgescited by 6
  35. 2022Bayesian Nonparametrics for Offline Skill Discoverycited by 5
  36. 2022Multi-Sample Training for Neural Image Compressioncited by 5
  37. 2024Stable Training of Normalizing Flows for High-dimensional Variational Inferencecited by 5
  38. 2024Fast and Unified Path Gradient Estimators for Normalizing Flowscited by 5

50 most cited of 97 All 97 (JSON)