# PAVI: Plate-Amortized Variational Inference

arXiv: https://arxiv.org/abs/2206.05111 · HTML: https://mlsanity.com/paper/2206.05111/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 66 references that are in the map.
- First posted on arXiv 10 Jun 2022.

## Builds on (most cited first, 28 of 28)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Masked Autoregressive Flow for Density Estimation](https://mlsanity.com/paper/1705.07057/) (2017)
- [Advances in Variational Inference](https://mlsanity.com/paper/1711.05597/) (2017)
- [Automatic Differentiation Variational Inference](https://mlsanity.com/paper/1603.00788/) (2016)
- [Inference Suboptimality in Variational Autoencoders](https://mlsanity.com/paper/1801.03558/) (2018)
- [Validating Bayesian Inference Algorithms with Simulation-Based Calibration](https://mlsanity.com/paper/1804.06788/) (2018)
- [TensorFlow Distributions](https://mlsanity.com/paper/1711.10604/) (2017)
- [Recent Advances in Recurrent Neural Networks](https://mlsanity.com/paper/1801.01078/) (2017)
- [Machine Learning for Neuroimaging with Scikit-Learn](https://mlsanity.com/paper/1412.3919/) (2014)
- [Hierarchically Structured Meta-learning](https://mlsanity.com/paper/1905.05301/) (2019)
- [Semi-Implicit Variational Inference](https://mlsanity.com/paper/1805.11183/) (2018)
- [Fast ε-free Inference of Simulation Models with Bayesian Conditional Density Estimation](https://mlsanity.com/paper/1605.06376/) (2016)
- [Unbiased Implicit Variational Inference](https://mlsanity.com/paper/1808.02078/) (2018)
- [Faithful Inversion of Generative Models for Effective Amortized Inference](https://mlsanity.com/paper/1712.00287/) (2017)
- [Fine-grain atlases of functional modes for fMRI analysis](https://mlsanity.com/paper/2003.05405/) (2020)
- [Structured Stochastic Variational Inference](https://mlsanity.com/paper/1404.4114/) (2014)
- [Automatic structured variational inference](https://mlsanity.com/paper/2002.00643/) (2020)
- [A Contrastive Divergence for Combining Variational Inference and MCMC](https://mlsanity.com/paper/1905.04062/) (2019)
- [Amortized Variational Inference for Simple Hierarchical Models](https://mlsanity.com/paper/2111.03144/) (2021)
- [Meta-Learning with Shared Amortized Variational Inference](https://mlsanity.com/paper/2008.12037/) (2020)
- [Automatic variational inference with cascading flows](https://mlsanity.com/paper/2102.04801/) (2021)
- [ADAVI: Automatic Dual Amortized Variational Inference Applied To Pyramidal Bayesian Models](https://mlsanity.com/paper/2106.12248/) (2021)

## Built on by (most cited first, 2 of 2)

- [Disentangling impact of capacity, objective, batchsize, estimators, and step-size on flow VI](https://mlsanity.com/paper/2412.08824/) (2024)
- [Semi-Amortized Bayesian Inference with Reusable Local Posteriors](https://mlsanity.com/paper/2610.18945/) (2026)

## Abstract (arXiv)

Given some observed data and a probabilistic generative model, Bayesian inference aims at obtaining the distribution of a model's latent parameters that could have yielded the data. This task is challenging for large population studies where thousands of measurements are performed over a cohort of hundreds of subjects, resulting in a massive latent parameter space. This large cardinality renders off-the-shelf Variational Inference (VI) computationally impractical. In this work, we design structured VI families that can efficiently tackle large population studies. To this end, our main idea is to share the parameterization and learning across the different i.i.d. variables in a generative model -symbolized by the model's plates. We name this concept plate amortization, and illustrate the powerful synergies it entitles, resulting in expressive, parsimoniously parameterized and orders of magnitude faster to train large scale hierarchical variational distributions. We illustrate the practical utility of PAVI through a challenging Neuroimaging example featuring a million latent parameters, demonstrating a significant step towards scalable and expressive Variational Inference.