# Probabilistic Autoencoder

arXiv: https://arxiv.org/abs/2006.05479 · HTML: https://mlsanity.com/paper/2006.05479/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jun 2020.

## Built on by (most cited first, 13 of 13)

- [Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connections](https://mlsanity.com/paper/2404.02954/) (2024)
- [Styleformer: Transformer based Generative Adversarial Networks with Style Vector](https://mlsanity.com/paper/2106.07023/) (2021)
- [Lifting Architectural Constraints of Injective Flows](https://mlsanity.com/paper/2306.01843/) (2023)
- [A Review of Change of Variable Formulas for Generative Modeling](https://mlsanity.com/paper/2308.02652/) (2023)
- [Latent Space Refinement for Deep Generative Models](https://mlsanity.com/paper/2106.00792/) (2021)
- [Model Selection for Bayesian Autoencoders](https://mlsanity.com/paper/2106.06245/) (2021)
- [VTAE: Variational Transformer Autoencoder with Manifolds Learning](https://mlsanity.com/paper/2304.00948/) (2023)
- [Likelihood-based Out-of-Distribution Detection with Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2310.17432/) (2023)
- [Deterministic training of generative autoencoders using invertible layers](https://mlsanity.com/paper/2205.09546/) (2022)
- [Interpretable Fake News Detection with Topic and Deep Variational Models](https://mlsanity.com/paper/2209.01536/) (2022)
- [Autoencoder Attractors for Uncertainty Estimation](https://mlsanity.com/paper/2204.00382/) (2022)
- [Classification and Uncertainty Quantification of Corrupted Data using Semi-Supervised Autoencoders](https://mlsanity.com/paper/2105.13393/) (2021)
- [NAE: Normalizing AutoEncoder](https://mlsanity.com/paper/2608.12084/) (2026)

## Abstract (arXiv)

Principal Component Analysis (PCA) minimizes the reconstruction error given a class of linear models of fixed component dimensionality. Probabilistic PCA adds a probabilistic structure by learning the probability distribution of the PCA latent space weights, thus creating a generative model. Autoencoders (AE) minimize the reconstruction error in a class of nonlinear models of fixed latent space dimensionality and outperform PCA at fixed dimensionality. Here, we introduce the Probabilistic Autoencoder (PAE) that learns the probability distribution of the AE latent space weights using a normalizing flow (NF). The PAE is fast and easy to train and achieves small reconstruction errors, high sample quality, and good performance in downstream tasks. We compare the PAE to Variational AE (VAE), showing that the PAE trains faster, reaches a lower reconstruction error, and produces good sample quality without requiring special tuning parameters or training procedures. We further demonstrate that the PAE is a powerful model for performing the downstream tasks of probabilistic image reconstruction in the context of Bayesian inference of inverse problems for inpainting and denoising applications. Finally, we identify latent space density from NF as a promising outlier detection metric.