# Partially Linear Autoencoders for Manifold Learning and Dimensionality Reduction

arXiv: https://arxiv.org/abs/2608.29867 · HTML: https://mlsanity.com/paper/2608.29867/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 46 references that are in the map.
- First posted on arXiv 30 Aug 2026.

## Builds on (most cited first, 17 of 17)

- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [The Intrinsic Dimension of Images and Its Impact on Learning](https://mlsanity.com/paper/2104.08894/) (2021)
- [Linearly-Recurrent Autoencoder Networks for Learning Dynamics](https://mlsanity.com/paper/1712.01378/) (2017)
- [Topological Autoencoders](https://mlsanity.com/paper/1906.00722/) (2019)
- [Parametric UMAP embeddings for representation and semi-supervised learning](https://mlsanity.com/paper/2009.12981/) (2020)
- [Manifold learning: what, how, and why](https://mlsanity.com/paper/2311.03757/) (2023)
- [A Survey on Masked Autoencoder for Self-supervised Learning in Vision and Beyond](https://mlsanity.com/paper/2208.00173/) (2022)
- [From Principal Subspaces to Principal Components with Linear Autoencoders](https://mlsanity.com/paper/1804.10253/) (2018)
- [Implicit Rank-Minimizing Autoencoder](https://mlsanity.com/paper/2010.00679/) (2020)
- [Isometric Autoencoders](https://mlsanity.com/paper/2006.09289/) (2020)
- [Autoencoders for discovering manifold dimension and coordinates in data from complex dynamical systems](https://mlsanity.com/paper/2305.01090/) (2023)
- [A Geometric Perspective on Autoencoders](https://mlsanity.com/paper/2309.08247/) (2023)
- [A Riemannian Framework for Learning Reduced-order Lagrangian Dynamics](https://mlsanity.com/paper/2410.18868/) (2024)
- [Convergent autoencoder approximation of low bending and low distortion manifold embeddings](https://mlsanity.com/paper/2208.10193/) (2022)
- [A Discussion On the Validity of Manifold Learning](https://mlsanity.com/paper/2106.01608/) (2021)
- [Are We Using Autoencoders in a Wrong Way?](https://mlsanity.com/paper/2309.01532/) (2023)
- [Recovery of Linear Components: Reduced Complexity Autoencoder Designs](https://mlsanity.com/paper/2012.07543/) (2020)

## Abstract (arXiv)

Autoencoders are widely used for nonlinear dimensionality reduction and manifold learning. While most common implementations rely on both nonlinear encoders and decoders, we investigate the specific role of the encoder and the extent to which it can be constrained to be linear without reducing accuracy. We conduct a comparative study on four autoencoder architectures: standard fully nonlinear autoencoders (AE), linear-encoder autoencoders (Lenc-AE), linear-decoder autoencoders (Ldec-AE), and fully linear autoencoders (LAE), evaluated on synthetic manifolds, computational mechanics data sets, and real-world image data sets including MNIST. We demonstrate that imposing a linear encoder preserves most of the representational capacity of the autoencoder, provided the decoder remains nonlinear. In particular, Lenc-AE consistently outperforms both Ldec-AE and LAE, and achieves reconstruction quality comparable to fully nonlinear AE, while offering advantages in terms of parsimony and interpretability of the latent representation. These results suggest that the nonlinear decoder is the critical component for manifold learning, rather than the encoder. A geometric interpretation of this finding is developed, which identifies the precise conditions under which a linear encoder is sufficient, and the specific manifold configurations that expose its limitations.