# A Unified Theory of Exact Inference and Learning in Exponential Family Latent Variable Models

arXiv: https://arxiv.org/abs/2404.19501 · HTML: https://mlsanity.com/paper/2404.19501/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 71 references that are in the map.
- First posted on arXiv 30 Apr 2024.

## Builds on (most cited first, 14 of 14)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [On Graphical Models via Univariate Exponential Family Distributions](https://mlsanity.com/paper/1301.4183/) (2013)
- [A State Space Approach for Piecewise-Linear Recurrent Neural Networks for Reconstructing Nonlinear Dynamics from Neural Measurements](https://mlsanity.com/paper/1612.07846/) (2016)
- [PCA Reduced Gaussian Mixture Models with Applications in Superresolution](https://mlsanity.com/paper/2009.07520/) (2020)
- [VAE Approximation Error: ELBO and Exponential Families](https://mlsanity.com/paper/2102.09310/) (2021)
- [Flexible and accurate inference and learning for deep generative models](https://mlsanity.com/paper/1805.11051/) (2018)
- [Approximating Bayes in the 21st Century](https://mlsanity.com/paper/2112.10342/) (2021)
- [Vector-Space Markov Random Fields via Exponential Families](https://mlsanity.com/paper/1505.05117/) (2015)
- [A probabilistic population code based on neural samples](https://mlsanity.com/paper/1811.09739/) (2018)
- [Riemann-Theta Boltzmann Machine](https://mlsanity.com/paper/1712.07581/) (2017)
- [Conditional Finite Mixtures of Poisson Distributions for Context-Dependent Neural Correlations](https://mlsanity.com/paper/1908.00637/) (2019)

## Abstract (arXiv)

Bayes' rule describes how to infer posterior beliefs about latent variables given observations, and inference is a critical step in learning algorithms for latent variable models (LVMs). Although there are exact algorithms for inference and learning for certain LVMs such as linear Gaussian models and mixture models, researchers must typically develop approximate inference and learning algorithms when applying novel LVMs. Here we study the line that separates LVMs that rely on approximation schemes from those that do not, and develop a general theory of exponential family LVMs for which inference and learning may be implemented exactly. Firstly, under mild assumptions about the exponential family form of the LVM, we derive a necessary and sufficient constraint on the parameters of the LVM under which the prior and posterior over the latent variables are in the same exponential family. We then show that a variety of well-known and novel models indeed have this constrained, exponential family form. Finally, we derive generalized inference and learning algorithms for these LVMs, and demonstrate them with a variety of examples. Our unified perspective facilitates both understanding and implementing exact inference and learning algorithms for a wide variety of models, and may guide researchers in the discovery of new models that avoid unnecessary approximations.