mlsanityData to 7 Oct 2026

Amortized Variational Inference for Simple Hierarchical Models

arXiv 2111.03144 ↗First posted 4 Nov 2021cs.LG stat.MLcited by 19

Not on the map. Why?

Abstract (arXiv)

It is difficult to use subsampling with variational inference in hierarchical models since the number of local latent variables scales with the dataset. Thus, inference in hierarchical models remains a challenge at large scale. It is helpful to use a variational family with structure matching the posterior, but optimization is still slow due to the huge number of local distributions. Instead, this paper suggests an amortized approach where shared parameters simultaneously represent all local distributions. This approach is similarly accurate as using a given joint distribution (e.g., a full-rank Gaussian) but is feasible on datasets that are several orders of magnitude larger. It is also dramatically faster than using a structured variational distribution.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 15 of its 40 references that are in the map. First posted on arXiv 4 Nov 2021.

Builds on

15 of 40 references.

References · 15

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Deep Setscited by 1,394
  3. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,315
  4. 2013Black Box Variational Inferencecited by 317
  5. 2016Automatic Differentiation Variational Inferencecited by 159
  6. 2016Towards a Neural Statisticiancited by 119
  7. 2019DIVA: Domain Invariant Variational Autoencoderscited by 114
  8. 2016Composing graphical models with neural networks for structured representations and fast inferencecited by 113
  9. 2017Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observationscited by 100
  10. 2017Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inferencecited by 97
  11. 2020Advances in Black-Box VI: Normalizing Flows, Importance Weighting, and Optimizationcited by 37
  12. 2019Provable Smoothness Guarantees for Black-Box Variational Inferencecited by 29
3 more
  1. 2020Automatic structured variational inferencecited by 21
  2. 2016Monte Carlo Structured SVI for Two-Level Non-Conjugate Modelscited by 3
  3. 2020What are the most important statistical ideas of the past 50 years?cited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 7; 2024: 6; 2025: 2; 2026: 2.

19 papers.

Sort:

By area:

  1. 2024Improved off-policy training of diffusion samplerscited by 47
  2. 2026Alternating Reinforcement Learning for Rubric-Based Reward Modeling in Non-Verifiable LLM Post-Trainingcited by 45
  3. 2023GFlowNet-EM for learning compositional latent variable modelscited by 40
  4. 2023Amortized Variational Inference: When and Why?cited by 28
  5. 2023On the Convergence of Black-Box Variational Inferencecited by 22
  6. 2024Amortized Bayesian Multilevel Modelscited by 12
  7. 2023The Shrinkage-Delinkage Trade-off: An Analysis of Factorized Gaussian Approximations for Variational Inferencecited by 9
  8. 2023Hierarchical Neural Simulation-Based Inference Over Event Ensemblescited by 8
  9. 2022Variational Inference with Locally Enhanced Bounds for Hierarchical Modelscited by 4
  10. 2024Provably Scalable Black-Box Variational Inference with Structured Variational Familiescited by 4
  11. 2024Fully Bayesian Differential Gaussian Processes through Stochastic Differential Equationscited by 4
  12. 2025Diffusion Bridge Variational Inference for Deep Gaussian Processescited by 4
7 more
  1. 2023Gaussian Process-Gated Hierarchical Mixtures of Expertscited by 2
  2. 2022PAVI: Plate-Amortized Variational Inferencecited by 1
  3. 2023PAVI: Plate-Amortized Variational Inferencecited by 1
  4. 2024Understanding and mitigating difficulties in posterior predictive evaluationcited by 1
  5. 2024Amortized Variational Inference for Deep Gaussian Processescited by 1
  6. 2025Massively Parallel Expectation Maximization For Approximate Posteriorscited by 0
  7. 2026VariBASed: Variational Bayes-Adaptive Sequential Monte-Carlo Planning for Deep Reinforcement Learningcited by 0