# Distributional Principal Autoencoders

arXiv: https://arxiv.org/abs/2404.13649 · HTML: https://mlsanity.com/paper/2404.13649/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 52 references that are in the map.
- First posted on arXiv 21 Apr 2024.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [Variational Autoencoders and Nonlinear ICA: A Unifying Framework](https://mlsanity.com/paper/1907.04809/) (2019)
- [Wasserstein Auto-Encoders](https://mlsanity.com/paper/1711.01558/) (2017)
- [Adversarial Autoencoders](https://mlsanity.com/paper/1511.05644/) (2015)
- [Adversarial Feature Learning](https://mlsanity.com/paper/1605.09782/) (2016)
- [Adversarially Learned Inference](https://mlsanity.com/paper/1606.00704/) (2016)
- [Flows for simultaneous manifold learning and density estimation](https://mlsanity.com/paper/2003.13913/) (2020)
- [Generalized Denoising Auto-Encoders as Generative Models](https://mlsanity.com/paper/1305.6663/) (2013)
- [Learning Ordered Representations with Nested Dropout](https://mlsanity.com/paper/1402.0915/) (2014)
- [Identifiability of deep generative models without auxiliary information](https://mlsanity.com/paper/2206.10044/) (2022)
- [Engression: Extrapolation through the Lens of Distributional Regression](https://mlsanity.com/paper/2307.00835/) (2023)
- [PCAAE: Principal Component Analysis Autoencoder for organising the latent space of generative networks](https://mlsanity.com/paper/2006.07827/) (2020)
- [Bidirectional Generative Modeling Using Adversarial Gradient Estimation](https://mlsanity.com/paper/2002.09161/) (2020)
- [Triangular Dropout: Variable Network Width without Retraining](https://mlsanity.com/paper/2205.01235/) (2022)
- [Information-Ordered Bottlenecks for Adaptive Semantic Compression](https://mlsanity.com/paper/2305.11213/) (2023)

## Built on by (most cited first, 11 of 11)

- [Distributional Autoencoders Know the Score](https://mlsanity.com/paper/2502.11583/) (2025)
- [Causality-Inspired Robustness for Nonlinear Models via Representation Learning](https://mlsanity.com/paper/2505.12868/) (2025)
- [Energy-Tweedie: Score meets Score, Energy meets Energy](https://mlsanity.com/paper/2512.23818/) (2025)
- [Anchor PCA](https://mlsanity.com/paper/2606.06233/) (2026)
- [Belted Engression: Sufficient Dimension Reduction for Generative Distributional Regression](https://mlsanity.com/paper/2609.23789/) (2026)
- [Extrapolation Guarantees for Perturbation Modeling Under the Additive Latent Shift Assumption](https://mlsanity.com/paper/2504.18522/) (2025)
- [Bilateral Distribution Compression: Reducing Both Data Size and Dimensionality](https://mlsanity.com/paper/2509.17543/) (2025)
- [Do Vision Transformers Need All-to-All Attention? Global Communication Through Elastic Learned Cores](https://mlsanity.com/paper/2605.12491/) (2026)
- [Distributional Extrapolation for Interactions](https://mlsanity.com/paper/2608.19849/) (2026)
- [Conditional-Independence-Regularized Distributional Autoencoders for Mixed-Type Data](https://mlsanity.com/paper/2608.20562/) (2026)
- [Sufficiently Reduced Distributional Regression](https://mlsanity.com/paper/2609.29291/) (2026)

## Abstract (arXiv)

Dimension reduction techniques usually lose information in the sense that reconstructed data are not identical to the original data. However, we argue that it is possible to have reconstructed data identically distributed as the original data, irrespective of the retained dimension or the specific mapping. This can be achieved by learning a distributional model that matches the conditional distribution of data given its low-dimensional latent variables. Motivated by this, we propose Distributional Principal Autoencoder (DPA) that consists of an encoder that maps high-dimensional data to low-dimensional latent variables and a decoder that maps the latent variables back to the data space. For reducing the dimension, the DPA encoder aims to minimise the unexplained variability of the data with an adaptive choice of the latent dimension. For reconstructing data, the DPA decoder aims to match the conditional distribution of all data that are mapped to a certain latent value, thus ensuring that the reconstructed data retains the original data distribution. Our numerical results on climate data, single-cell data, and image benchmarks demonstrate the practical feasibility and success of the approach in reconstructing the original distribution of the data. DPA embeddings are shown to preserve meaningful structures of data such as the seasonal cycle for precipitations and cell types for gene expression.