mlsanityData to 14 Oct 2026

Automatic Differentiation Variational Inference

arXiv 1603.00788 ↗First posted 2 Mar 2016stat.ML cs.AI cs.LGcited by 159

Not on the map. Why?

Abstract (arXiv)

Probabilistic modeling is iterative. A scientist posits a simple model, fits it to her data, refines it according to her analysis, and repeats. However, fitting complex models to large data is a bottleneck in this process. Deriving algorithms for new models can be both mathematically and computationally challenging, which makes it difficult to efficiently cycle through the steps. To this end, we develop automatic differentiation variational inference (ADVI). Using our method, the scientist only provides a probabilistic model and a dataset, nothing else. ADVI automatically derives an efficient variational inference algorithm, freeing the scientist to refine and explore many models. ADVI supports a broad class of models-no conjugacy assumptions are required. We study ADVI across ten different models and apply it to a dataset with millions of observations. ADVI is integrated into Stan, a probabilistic programming system; it is available for immediate use.

Key facts. Cited by 159 papers in the map (data through 14 Oct 2026). First posted on arXiv 2 Mar 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 1; 2021: 26; 2022: 20; 2023: 33; 2024: 31; 2025: 19; 2026: 29.

159 papers.

Sort:

By area:

  1. 2023Provable convergence guarantees for black-box variational inferencecited by 29
  2. 2021Pathfinder: Parallel quasi-Newton variational inferencecited by 26
  3. 2023On the Convergence of Black-Box Variational Inferencecited by 22
  4. 2024Batch and match: black-box variational inference with a score-based divergencecited by 22
  5. 2026EdgeBench: Unveiling Scaling Laws of Learning from Real-World Environmentscited by 20
  6. 2021Amortized Variational Inference for Simple Hierarchical Modelscited by 19
  7. 2022Mean-field Variational Inference via Wasserstein Gradient Flowcited by 19
  8. 2023Black Box Variational Inference with a Deterministic Objective: Faster, More Accurate, and Even More Black Boxcited by 17
  9. 2023Variational Inference with Gaussian Score Matchingcited by 17
  10. 2023Fortuna: A Library for Uncertainty Quantification in Deep Learningcited by 14
  11. 2022Partitioned Variational Inference: A Framework for Probabilistic Federated Learningcited by 13
  12. 2021Geometric variational inferencecited by 12
38 more
  1. 2022Bayesian inference via sparse Hamiltonian flowscited by 12
  2. 2022Fat-Tailed Variational Inference with Anisotropic Tail Adaptive Flowscited by 11
  3. 2024Variational Inference for Uncertainty Quantification: an Analysis of Trade-offscited by 11
  4. 2024EigenVI: score-based variational inference with orthogonal function expansionscited by 11
  5. 2023The Shrinkage-Delinkage Trade-off: An Analysis of Factorized Gaussian Approximations for Variational Inferencecited by 10
  6. 2023Linear Convergence of Black-Box Variational Inference: Should We Stick the Landing?cited by 10
  7. 2023Partially factorized variational inference for high-dimensional mixed modelscited by 10
  8. 2024Optimal Classification under Performative Distribution Shiftcited by 10
  9. 2021Automatic variational inference with cascading flowscited by 9
  10. 2021ARMS: Antithetic-REINFORCE-Multi-Sample Gradient for Binary Variablescited by 9
  11. 2022Hybridizing Physical and Data-driven Prediction Methods for Physicochemical Propertiescited by 9
  12. 2024Variational Inference in Location-Scale Families: Exact Recovery of the Mean and Correlation Matrixcited by 9
  13. 2021The computational asymptotics of Gaussian variational inference and the Laplace approximationcited by 8
  14. 2021Breiman's two cultures: You don't have to choose sidescited by 8
  15. 2024Batch, match, and patch: low-rank approximations for score-based variational inferencecited by 8
  16. 2021BDNNSurv: Bayesian deep neural networks for survival analysis using pseudo valuescited by 7
  17. 2023Meta-learning Control Variates: Variance Reduction with Limited Datacited by 7
  18. 2024Particle Semi-Implicit Variational Inferencecited by 7
  19. 2024What's the score? Automated Denoising Score Matching for Nonlinear Diffusionscited by 7
  20. 2022Foundation Posteriors for Approximate Probabilistic Inferencecited by 6
  21. 2022Markov Chain Score Ascent: A Unifying Framework of Variational Inference with Markovian Gradientscited by 6
  22. 2022Cooperation in the Latent Space: The Benefits of Adding Mixture Components in Variational Autoencoderscited by 6
  23. 2023Practical and Matching Gradient Variance Bounds for Black-Box Variational Bayesian Inferencecited by 6
  24. 2023PriorCVAE: scalable MCMC parameter inference with Bayesian deep generative modellingcited by 6
  25. 2023Sample Average Approximation for Black-Box VIcited by 6
  26. 2023Embracing the chaos: analysis and diagnosis of numerical instability in variational flowscited by 6
  27. 2024Stable Training of Normalizing Flows for High-dimensional Variational Inferencecited by 6
  28. 2024On Divergence Measures for Training GFlowNetscited by 6
  29. 2025Variational Inference with Mixtures of Isotropic Gaussianscited by 6
  30. 2021Variational Inference for Deblending Crowded Starfieldscited by 5
  31. 2021How Bayesian Should Bayesian Optimisation Be?cited by 5
  32. 2023Learning to solve Bayesian inverse problems: An amortized variational inference approach using Gaussian and Flow guidescited by 5
  33. 2024Annealed Stein Variational Gradient Descent for Improved Uncertainty Estimation in Full-Waveform Inversioncited by 5
  34. 2021ADAVI: Automatic Dual Amortized Variational Inference Applied To Pyramidal Bayesian Modelscited by 4
  35. 2021Evaluating Sensitivity to the Stick-Breaking Prior in Bayesian Nonparametricscited by 4
  36. 2021CARMS: Categorical-Antithetic-REINFORCE Multi-Sample Gradient Estimatorcited by 4
  37. 2023Automatically Marginalized MCMC in Probabilistic Programmingcited by 4
  38. 2023Bayesian Pseudo-Coresets via Contrastive Divergencecited by 4

50 most cited of 159 All 159 (JSON)