# A Tutorial on VAEs: From Bayes' Rule to Lossless Compression

arXiv: https://arxiv.org/abs/2006.10273 · HTML: https://mlsanity.com/paper/2006.10273/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jun 2020.

## Built on by (most cited first, 11 of 11)

- [3DLinker: An E(3) Equivariant Variational Autoencoder for Molecular Linker Design](https://mlsanity.com/paper/2205.07309/) (2022)
- [Efficient-VDVAE: Less is more](https://mlsanity.com/paper/2203.13751/) (2022)
- [VENI, VINDy, VICI: a generative reduced-order modeling framework with uncertainty quantification](https://mlsanity.com/paper/2405.20905/) (2024)
- [Generating Multivariate Load States Using a Conditional Variational Autoencoder](https://mlsanity.com/paper/2110.11435/) (2021)
- [Disentangling Generative Factors of Physical Fields Using Variational Autoencoders](https://mlsanity.com/paper/2109.07399/) (2021)
- [DA-VEGAN: Differentiably Augmenting VAE-GAN for microstructure reconstruction from extremely small data sets](https://mlsanity.com/paper/2303.03403/) (2023)
- [Interpreting Latent Spaces of Generative Models for Medical Images using Unsupervised Methods](https://mlsanity.com/paper/2207.09740/) (2022)
- [CR-VAE: Contrastive Regularization on Variational Autoencoders for Preventing Posterior Collapse](https://mlsanity.com/paper/2309.02968/) (2023)
- [Decouple to Reconstruct: High Quality UHD Restoration via Active Feature Disentanglement and Reversible Fusion](https://mlsanity.com/paper/2503.12764/) (2025)
- [Beta-Sigma VAE: Separating beta and decoder variance in Gaussian variational autoencoder](https://mlsanity.com/paper/2409.09361/) (2024)
- [Learning Disentangled Equivariant Representation for Explicitly Controllable 3D Molecule Generation](https://mlsanity.com/paper/2412.15086/) (2024)

## Abstract (arXiv)

The Variational Auto-Encoder (VAE) is a simple, efficient, and popular deep maximum likelihood model. Though usage of VAEs is widespread, the derivation of the VAE is not as widely understood. In this tutorial, we will provide an overview of the VAE and a tour through various derivations and interpretations of the VAE objective. From a probabilistic standpoint, we will examine the VAE through the lens of Bayes' Rule, importance sampling, and the change-of-variables formula. From an information theoretic standpoint, we will examine the VAE through the lens of lossless compression and transmission through a noisy channel. We will then identify two common misconceptions over the VAE formulation and their practical consequences. Finally, we will visualize the capabilities and limitations of VAEs using a code example (with an accompanying Jupyter notebook) on toy 2D data.