mlsanityData to 7 Oct 2026

Variational Inference: A Review for Statisticians

arXiv 1601.00670 ↗First posted 4 Jan 2016stat.CO cs.LG stat.MLcited by 1,466

Not on the map. Why?

Cited by 53 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

One of the core problems of modern statistics is to approximate difficult-to-compute probability densities. This problem is especially important in Bayesian statistics, which frames all inference about unknown quantities as a calculation involving the posterior density. In this paper, we review variational inference (VI), a method from machine learning that approximates probability densities through optimization. VI has been used in many applications and tends to be faster than classical methods, such as Markov chain Monte Carlo sampling. The idea behind VI is to first posit a family of densities and then to find the member of that family which is close to the target. Closeness is measured by Kullback-Leibler divergence. We review the ideas behind mean-field variational inference, discuss the special case of VI applied to exponential family models, present a full example with a Bayesian mixture of Gaussians, and derive a variant that uses stochastic optimization to scale up to massive data. We discuss modern research in VI and highlight important open problems. VI is powerful, but it is not yet well understood. Our hope in writing this paper is to catalyze statistical research on this class of algorithms.

Key facts. Cited by 1,466 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Jan 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 1; 2021: 245; 2022: 247; 2023: 255; 2024: 272; 2025: 243; 2026: 202.

1,466 papers.

Sort:

By area:

  1. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 543
  2. 2021Transformers Can Do Bayesian Inferencecited by 338
  3. 2023Your Diffusion Model is Secretly a Zero-Shot Classifiercited by 301
  4. 2021User-friendly introduction to PAC-Bayes boundscited by 234
  5. 2023A Variational Perspective on Solving Inverse Problems with Diffusion Modelscited by 217
  6. 2021D'ya like DAGs? A Survey on Structure Learning and Causal Discoverycited by 209
  7. 2022Diffusion-GAN: Training GANs with Diffusioncited by 203
  8. 2022Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisonscited by 169
  9. 2024Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learningcited by 147
  10. 2022Self-supervised Learning in Remote Sensing: A Reviewcited by 137
  11. 2022RL with KL penalties is better viewed as Bayesian inferencecited by 130
  12. 2021KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamicscited by 128
38 more
  1. 2023Score-Based Diffusion Models as Principled Priors for Inverse Imagingcited by 126
  2. 2023Modern Bayesian Experimental Designcited by 121
  3. 2021DEUP: Direct Epistemic Uncertainty Predictioncited by 102
  4. 2021Priors in Bayesian Deep Learning: A Reviewcited by 101
  5. 2023Towards Zero-Shot Scale-Aware Monocular Depth Estimationcited by 100
  6. 2021The Causal-Neural Connection: Expressiveness, Learnability, and Inferencecited by 97
  7. 2021The Bayesian Learning Rulecited by 95
  8. 2022Deep End-to-end Causal Inferencecited by 93
  9. 2022Amortized Inference for Causal Structure Learningcited by 89
  10. 2022Variational inference via Wasserstein gradient flowscited by 84
  11. 2022Tutorial on amortized optimizationcited by 83
  12. 2021Stochastic Neural Radiance Fields: Quantifying Uncertainty in Implicit 3D Representationscited by 80
  13. 2023A Survey on Uncertainty Quantification Methods for Deep Learningcited by 78
  14. 2021Large-Scale Wasserstein Gradient Flowscited by 76
  15. 2021Desiderata for Representation Learning: A Causal Perspectivecited by 76
  16. 2023Diffusion Model is Secretly a Training-free Open Vocabulary Semantic Segmentercited by 76
  17. 2022Variational Model Inversion Attackscited by 75
  18. 2021Predictive Coding: a Theoretical and Experimental Reviewcited by 74
  19. 2021A Survey on Graph Structure Learning: Progress and Opportunitiescited by 70
  20. 2021BCD Nets: Scalable Variational Approaches for Bayesian Causal Discoverycited by 68
  21. 2022Personalized Federated Learning via Variational Bayesian Inferencecited by 68
  22. 2023Recent Advances in Optimal Transport for Machine Learningcited by 66
  23. 2024Autoregressive Speech Synthesis without Vector Quantizationcited by 65
  24. 2023GFlowNets for AI-Driven Scientific Discoverycited by 63
  25. 2025CUDA-L1: Improving CUDA Optimization via Contrastive Reinforcement Learningcited by 59
  26. 2023Posterior Collapse and Latent Variable Non-identifiabilitycited by 57
  27. 2023Diffusion Generative Flow Samplers: Improving learning signals through partial trajectory optimizationcited by 57
  28. 2021Scalable Bayesian Inverse Reinforcement Learningcited by 56
  29. 2023Learning Mixtures of Gaussians Using the DDPM Objectivecited by 56
  30. 2024Beyond ELBOs: A Large-Scale Evaluation of Variational Methods for Samplingcited by 55
  31. 2021Instance-dependent Label-noise Learning under a Structural Causal Modelcited by 50
  32. 2022Sequential Neural Score Estimation: Likelihood-Free Inference with Conditional Score Based Diffusion Modelscited by 50
  33. 2021GIBBON: General-purpose Information-Based Bayesian OptimisatioNcited by 47
  34. 2021Task-Oriented Communication for Multi-Device Cooperative Edge Inferencecited by 47
  35. 2022Probability flow solution of the Fokker-Planck equationcited by 46
  36. 2022Uncertainty-guided Source-free Domain Adaptationcited by 46
  37. 2023Effective Neural Topic Modeling with Embedding Clustering Regularizationcited by 46
  38. 2021Posterior Meta-Replay for Continual Learningcited by 45

50 most cited of 1,466 All 1,466 (JSON)