mlsanityData to 7 Oct 2026

Amortized Variational Inference: A Systematic Review

arXiv 2209.10888 ↗First posted 22 Sep 2022cs.LG cs.IT math.ITcited by 16

Not on the map. Why?

Abstract (arXiv)

The core principle of Variational Inference (VI) is to convert the statistical inference problem of computing complex posterior probability densities into a tractable optimization problem. This property enables VI to be faster than several sampling-based techniques. However, the traditional VI algorithm is not scalable to large data sets and is unable to readily infer out-of-bounds data points without re-running the optimization process. Recent developments in the field, like stochastic-, black box-, and amortized-VI, have helped address these issues. Generative modeling tasks nowadays widely make use of amortized VI for its efficiency and scalability, as it utilizes a parameterized function to learn the approximate posterior density parameters. In this paper, we review the mathematical foundations of various VI techniques to form the basis for understanding amortized VI. Additionally, we provide an overview of the recent trends that address several issues of amortized VI, such as the amortization gap, generalization issues, inconsistent representation learning, and posterior collapse. Finally, we analyze alternate divergence measures that improve VI optimization.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 77 of its 163 references that are in the map. First posted on arXiv 22 Sep 2022.

Builds on

77 of 163 references.

References · 77

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  4. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  5. 2021Convex optimizationcited by 2,479
  6. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  7. 2015Variational Inference with Normalizing Flowscited by 2,161
  8. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  9. 2016Variational Inference: A Review for Statisticianscited by 1,466
  10. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
  11. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 858
  12. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
38 more
  1. 2016Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networkscited by 541
  2. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 498
  3. 2015Importance Weighted Autoencoderscited by 454
  4. 2014New insights and perspectives on the natural gradient methodcited by 435
  5. 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
  6. 2017Wasserstein Auto-Encoderscited by 371
  7. 2018Understanding disentangling in β-VAEcited by 332
  8. 2013Black Box Variational Inferencecited by 317
  9. 2016Ladder Variational Autoencoderscited by 315
  10. 2018Variational Autoencoders for Collaborative Filteringcited by 308
  11. 2019Monte Carlo Gradient Estimation in Machine Learningcited by 281
  12. 2016A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluationcited by 236
  13. 2017Advances in Variational Inferencecited by 236
  14. 2016Variational Lossy Autoencodercited by 175
  15. 2013Expectation Propagation for approximate Bayesian inferencecited by 172
  16. 2020Adversarial Latent Autoencoderscited by 155
  17. 2016A Kernel Test of Goodness of Fitcited by 143
  18. 2018Inference Suboptimality in Variational Autoencoderscited by 131
  19. 2018Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adamcited by 131
  20. 2019Functional Variational Bayesian Neural Networkscited by 127
  21. 2016R'enyi Divergence Variational Inferencecited by 104
  22. 2018Iterative Amortized Inferencecited by 93
  23. 2014Stochastic Optimization with Importance Samplingcited by 91
  24. 2015Hierarchical Variational Modelscited by 89
  25. 2019Preventing Posterior Collapse with delta-VAEscited by 86
  26. 2018Semi-Amortized Variational Autoencoderscited by 83
  27. 2022Tutorial on amortized optimizationcited by 83
  28. 2017Improved Variational Autoencoders for Text Modeling using Dilated Convolutionscited by 81
  29. 2013Loopy Belief Propagation for Approximate Inference: An Empirical Studycited by 76
  30. 2018Tighter Variational Bounds are Not Necessarily Bettercited by 76
  31. 2017Metrics for Deep Generative Modelscited by 53
  32. 2021Consistency Regularization for Variational Auto-Encoderscited by 50
  33. 2016Stick-Breaking Variational Autoencoderscited by 44
  34. 2015Denoising Criterion for Variational Auto-Encoding Frameworkcited by 43
  35. 2017Stein Variational Policy Gradientcited by 42
  36. 2018Fast yet Simple Natural-Gradient Descent for Variational Inference in Complex Modelscited by 37
  37. 2016Importance Sampling for Minibatchescited by 36
  38. 2021Monte Carlo Variational Auto-Encoderscited by 36

50 most cited of 77 All 77 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 6; 2025: 6; 2026: 3.

16 papers.

Sort:

By area:

  1. 2024Neural Methods for Amortized Inferencecited by 36
  2. 2024Diffusion Prior-Based Amortized Variational Inference for Noisy Inverse Problemscited by 13
  3. 2025Distribution Transformers: Fast Approximate Bayesian Inference With On-The-Fly Prior Adaptationcited by 12
  4. 2024Poisson Variational Autoencodercited by 11
  5. 2025Probabilistic operator learning: generative modeling and uncertainty quantification for foundation models of differential equationscited by 9
  6. 2024Compositional simulation-based inference for time seriescited by 8
  7. 2023Learning to solve Bayesian inverse problems: An amortized variational inference approach using Gaussian and Flow guidescited by 5
  8. 2025DGenNO: A Novel Physics-aware Neural Operator for Solving Forward and Inverse PDE Problems based on Deep, Generative Probabilistic Modelingcited by 5
  9. 2024Active Inference and Human--Computer Interactioncited by 4
  10. 2025Diffusion Bridge Variational Inference for Deep Gaussian Processescited by 4
  11. 2025Sparse Gaussian Neural Processescited by 2
  12. 2024Amortized Variational Inference for Deep Gaussian Processescited by 1
4 more
  1. 2025Meta-Statistical Learning: Supervised Learning of Statistical Estimatorscited by 1
  2. 2026Slack More, Predict Better: Proximal Relaxation for Probabilistic Latent Variable Model-based Soft Sensorscited by 0
  3. 2026Amortized Variational Inference for Logistic Regression with Missing Covariatescited by 0
  4. 2026Instance-Adaptive Parametrization for Amortized Variational Inferencecited by 0