# On the Necessity and Effectiveness of Learning the Prior of Variational Auto-Encoder

arXiv: https://arxiv.org/abs/1905.13452 · HTML: https://mlsanity.com/paper/1905.13452/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 May 2019.

## Built on by (most cited first, 6 of 6)

- [Generative Causal Representation Learning for Out-of-Distribution Motion Forecasting](https://mlsanity.com/paper/2302.08635/) (2023)
- [Weakly Supervised Bayesian Shape Modeling from Unsegmented Medical Images](https://mlsanity.com/paper/2405.09697/) (2024)
- [Improving Variational Autoencoders with Density Gap-based Regularization](https://mlsanity.com/paper/2211.00321/) (2022)
- [A Contextual Latent Space Model: Subsequence Modulation in Melodic Sequence](https://mlsanity.com/paper/2111.11703/) (2021)
- [RENs: Relevance Encoding Networks](https://mlsanity.com/paper/2205.13061/) (2022)
- [Learning Nonparametric High-Dimensional Generative Models: The Empirical-Beta-Copula Autoencoder](https://mlsanity.com/paper/2309.09916/) (2023)

## Abstract (arXiv)

Using powerful posterior distributions is a popular approach to achieving better variational inference. However, recent works showed that the aggregated posterior may fail to match unit Gaussian prior, thus learning the prior becomes an alternative way to improve the lower-bound. In this paper, for the first time in the literature, we prove the necessity and effectiveness of learning the prior when aggregated posterior does not match unit Gaussian prior, analyze why this situation may happen, and propose a hypothesis that learning the prior may improve reconstruction loss, all of which are supported by our extensive experiment results. We show that using learned Real NVP prior and just one latent variable in VAE, we can achieve test NLL comparable to very deep state-of-the-art hierarchical VAE, outperforming many previous works with complex hierarchical VAE architectures.