# A Generalised Linear Model Framework for β-Variational Autoencoders based on Exponential Dispersion Families

arXiv: https://arxiv.org/abs/2006.06267 · HTML: https://mlsanity.com/paper/2006.06267/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jun 2020.

## Built on by (most cited first, 7 of 7)

- [Multi-Rate VAE: Train Once, Get the Full Rate-Distortion Curve](https://mlsanity.com/paper/2212.03905/) (2022)
- [VAE Approximation Error: ELBO and Exponential Families](https://mlsanity.com/paper/2102.09310/) (2021)
- [Learning Dynamics in Linear VAE: Posterior Collapse Threshold, Superfluous Latent Space Pitfalls, and Speedup with KL Annealing](https://mlsanity.com/paper/2310.15440/) (2023)
- [High-dimensional Asymptotics of VAEs: Threshold of Posterior Collapse and Dataset-Size Dependence of Rate-Distortion Curve](https://mlsanity.com/paper/2309.07663/) (2023)
- [Causally Disentangled Generative Variational AutoEncoder](https://mlsanity.com/paper/2302.11737/) (2023)
- [Flexible Variational Information Bottleneck: Achieving Diverse Compression with a Single Training](https://mlsanity.com/paper/2402.01238/) (2024)
- [Estimating the Value-at-Risk by Temporal VAE](https://mlsanity.com/paper/2112.01896/) (2021)

## Abstract (arXiv)

Although variational autoencoders (VAE) are successfully used to obtain meaningful low-dimensional representations for high-dimensional data, the characterization of critical points of the loss function for general observation models is not fully understood. We introduce a theoretical framework that is based on a connection between β-VAE and generalized linear models (GLM). The equality between the activation function of a β-VAE and the inverse of the link function of a GLM enables us to provide a systematic generalization of the loss analysis for β-VAE based on the assumption that the observation model distribution belongs to an exponential dispersion family (EDF). As a result, we can initialize β-VAE nets by maximum likelihood estimates (MLE) that enhance the training performance on both synthetic and real world data sets. As a further consequence, we analytically describe the auto-pruning property inherent in the β-VAE objective and reason for posterior collapse.