# Tutorial: Deriving the Standard Variational Autoencoder (VAE) Loss Function

arXiv: https://arxiv.org/abs/1907.08956 · HTML: https://mlsanity.com/paper/1907.08956/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jul 2019.

## Built on by (most cited first, 26 of 26)

- [Hybrid Predictive Coding: Inferring, Fast and Slow](https://mlsanity.com/paper/2204.02169/) (2022)
- [Factor Analysis, Probabilistic Principal Component Analysis, Variational Inference, and Variational Autoencoder: Tutorial and Survey](https://mlsanity.com/paper/2101.00734/) (2021)
- [Generalizing Across Domains in Diabetic Retinopathy via Variational Autoencoders](https://mlsanity.com/paper/2309.11301/) (2023)
- [Disentangling Generative Factors of Physical Fields Using Variational Autoencoders](https://mlsanity.com/paper/2109.07399/) (2021)
- [Paired Wasserstein Autoencoders for Conditional Sampling](https://mlsanity.com/paper/2412.07586/) (2024)
- [Variational Inference for Quantifying Inter-observer Variability in Segmentation of Anatomical Structures](https://mlsanity.com/paper/2201.07106/) (2022)
- [Deep Data Consistency: a Fast and Robust Diffusion Model-based Solver for Inverse Problems](https://mlsanity.com/paper/2405.10748/) (2024)
- [Balanced Neural ODEs: nonlinear model order reduction and Koopman operator approximations](https://mlsanity.com/paper/2410.10174/) (2024)
- [Unsupervised Clustering of Roman Potsherds via Variational Autoencoders](https://mlsanity.com/paper/2203.07437/) (2022)
- [Tackling Multimodal Device Distributions in Inverse Photonic Design using Invertible Neural Networks](https://mlsanity.com/paper/2208.14212/) (2022)
- [Data-driven Nonlinear Parametric Model Order Reduction Framework using Deep Hierarchical Variational Autoencoder](https://mlsanity.com/paper/2307.06816/) (2023)
- [Uncertainty-aware transfer across tasks using hybrid model-based successor feature reinforcement learning](https://mlsanity.com/paper/2310.10818/) (2023)
- [Energy-Based Prior Latent Space Diffusion model for Reconstruction of Lumbar Vertebrae from Thick Slice MRI](https://mlsanity.com/paper/2412.00511/) (2024)
- [Maximum Entropy Auto-Encoding](https://mlsanity.com/paper/2104.07448/) (2021)
- [Variational Autoencoder Assisted Neural Network Likelihood RSRP Prediction Model](https://mlsanity.com/paper/2207.00166/) (2022)
- [Variational Self-Supervised Contrastive Learning Using Beta Divergence](https://mlsanity.com/paper/2312.00824/) (2023)
- [Projected Belief Networks With Discriminative Alignment for Acoustic Event Classification: Rivaling State of the Art CNNs](https://mlsanity.com/paper/2401.11199/) (2024)
- [CF-OPT: Counterfactual Explanations for Structured Prediction](https://mlsanity.com/paper/2405.18293/) (2024)
- [Deep evolving semi-supervised anomaly detection](https://mlsanity.com/paper/2412.00860/) (2024)
- [Active Learning-Guided Seq2Seq Variational Autoencoder for Multi-target Inhibitor Generation](https://mlsanity.com/paper/2506.15309/) (2025)
- [Using KL-Divergence to Focus Frequency Information in Low-Light Image Enhancement](https://mlsanity.com/paper/2509.13083/) (2025)
- [Variational Contrastive Learning for Skeleton-based Action Recognition](https://mlsanity.com/paper/2601.07666/) (2026)
- [Cryo-SWAN: the Multi-Scale Wavelet-decomposition-inspired Autoencoder Network for molecular density representation of molecular volumes](https://mlsanity.com/paper/2603.03342/) (2026)
- [VAMP-Diff: VampPrior Latent Diffusion for Photoplethysmography Modeling](https://mlsanity.com/paper/2605.22851/) (2026)
- [Reconstructing and forecasting disease trajectories of patients with Alzheimer's disease using routine data in resource-constrained settings](https://mlsanity.com/paper/2606.07798/) (2026)
- [AI-Generated Image Recognition via Fusion of CNNs and Vision Transformers](https://mlsanity.com/paper/2606.27637/) (2026)

## Abstract (arXiv)

In Bayesian machine learning, the posterior distribution is typically computationally intractable, hence variational inference is often required. In this approach, an evidence lower bound on the log likelihood of data is maximized during training. Variational Autoencoders (VAE) are one important example where variational inference is utilized. In this tutorial, we derive the variational lower bound loss function of the standard variational autoencoder. We do so in the instance of a gaussian latent prior and gaussian approximate posterior, under which assumptions the Kullback-Leibler term in the variational lower bound has a closed form solution. We derive essentially everything we use along the way; everything from Bayes' theorem to the Kullback-Leibler divergence.