# ED-VAE: Entropy Decomposition of ELBO in Variational Autoencoders

arXiv: https://arxiv.org/abs/2407.06797 · HTML: https://mlsanity.com/paper/2407.06797/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 9 references that are in the map.
- First posted on arXiv 9 Jul 2024.

## Builds on (most cited first, 5 of 5)

- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Don't Blame the ELBO! A Linear VAE Perspective on Posterior Collapse](https://mlsanity.com/paper/1911.02469/) (2019)
- [Multiple Importance Sampling ELBO and Deep Ensembles of Variational Approximations](https://mlsanity.com/paper/2202.10951/) (2022)
- [Forget-me-not! Contrastive Critics for Mitigating Posterior Collapse](https://mlsanity.com/paper/2207.09535/) (2022)
- [CR-VAE: Contrastive Regularization on Variational Autoencoders for Preventing Posterior Collapse](https://mlsanity.com/paper/2309.02968/) (2023)

## Built on by (most cited first, 1 of 1)

- [Watermarking and Anomaly Detection in Machine Learning Models for LORA RF Fingerprinting](https://mlsanity.com/paper/2509.15170/) (2025)

## Abstract (arXiv)

Traditional Variational Autoencoders (VAEs) are constrained by the limitations of the Evidence Lower Bound (ELBO) formulation, particularly when utilizing simplistic, non-analytic, or unknown prior distributions. These limitations inhibit the VAE's ability to generate high-quality samples and provide clear, interpretable latent representations. This work introduces the Entropy Decomposed Variational Autoencoder (ED-VAE), a novel re-formulation of the ELBO that explicitly includes entropy and cross-entropy components. This reformulation significantly enhances model flexibility, allowing for the integration of complex and non-standard priors. By providing more detailed control over the encoding and regularization of latent spaces, ED-VAE not only improves interpretability but also effectively captures the complex interactions between latent variables and observed data, thus leading to better generative performance.