# PCAE: Learning Ordered Representations in Latent Space for Intrinsic Dimension Estimation via Principal Component Autoencoder

arXiv: https://arxiv.org/abs/2601.19179 · HTML: https://mlsanity.com/paper/2601.19179/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 19 references that are in the map.
- First posted on arXiv 27 Jan 2026.

## Builds on (most cited first, 9 of 9)

- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [The Intrinsic Dimension of Images and Its Impact on Learning](https://mlsanity.com/paper/2104.08894/) (2021)
- [Learning Ordered Representations with Nested Dropout](https://mlsanity.com/paper/1402.0915/) (2014)
- [Loss Landscapes of Regularized Linear Autoencoders](https://mlsanity.com/paper/1901.08168/) (2019)
- [From Principal Subspaces to Principal Components with Linear Autoencoders](https://mlsanity.com/paper/1804.10253/) (2018)
- [Implicit Rank-Minimizing Autoencoder](https://mlsanity.com/paper/2010.00679/) (2020)
- [Regularized linear autoencoders recover the principal components, eventually](https://mlsanity.com/paper/2007.06731/) (2020)
- [PCAAE: Principal Component Analysis Autoencoder for organising the latent space of generative networks](https://mlsanity.com/paper/2006.07827/) (2020)
- [ARD-VAE: A Statistical Formulation to Find the Relevant Latent Dimensions of Variational Autoencoders](https://mlsanity.com/paper/2501.10901/) (2025)

## Built on by (most cited first, 2 of 2)

- [A Theory of Speciation in Generative Diffusion Models on Compact Riemannian Manifolds](https://mlsanity.com/paper/2608.23798/) (2026)
- [Objective-Specific Privileged Bases via Full-Prefix Matryoshka Learning](https://mlsanity.com/paper/2605.09160/) (2026)

## Abstract (arXiv)

Autoencoders have long been considered a nonlinear extension of Principal Component Analysis (PCA). Prior studies have demonstrated that linear autoencoders (LAEs) can recover the ordered, axis-aligned principal components of PCA by incorporating non-uniform ℓ2 regularization or by adjusting the loss function. However, these approaches become insufficient in the nonlinear setting, as the remaining variance cannot be properly captured independently of the nonlinear mapping. In this work, we propose a novel autoencoder framework that integrates non-uniform variance regularization with an isometric constraint. This design serves as a natural generalization of PCA, enabling the model to preserve key advantages, such as ordered representations and variance retention, while remaining effective for nonlinear dimensionality reduction tasks.