# Rank Reduction Autoencoders

arXiv: https://arxiv.org/abs/2405.13980 · HTML: https://mlsanity.com/paper/2405.13980/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 23 references that are in the map.
- First posted on arXiv 22 May 2024.

## Builds on (most cited first, 8 of 8)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-based Variational Autoencoder](https://mlsanity.com/paper/1711.00614/) (2017)
- [Implicit Rank-Minimizing Autoencoder](https://mlsanity.com/paper/2010.00679/) (2020)
- [Regularized Autoencoders via Relaxed Injective Probability Flow](https://mlsanity.com/paper/2002.08927/) (2020)
- [Why Regularized Auto-Encoders learn Sparse Representation?](https://mlsanity.com/paper/1505.05561/) (2015)
- [An Autoencoder Approach to Learning Bilingual Word Representations](https://mlsanity.com/paper/1402.1454/) (2014)
- [Innovations Autoencoder and its Application in One-class Anomalous Sequence Detection](https://mlsanity.com/paper/2106.12382/) (2021)

## Built on by (most cited first, 5 of 5)

- [RRAEDy: Adaptive Latent Linearization of Nonlinear Dynamical Systems](https://mlsanity.com/paper/2512.07542/) (2025)
- [Data-driven discovery of roughness descriptors for surface characterization and intimate contact modeling of unidirectional composite tapes](https://mlsanity.com/paper/2603.20418/) (2026)
- [Optimal Linear Baseline Models for Scientific Machine Learning](https://mlsanity.com/paper/2508.05831/) (2025)
- [CRADIPOR: Crash Dispersion Predictor](https://mlsanity.com/paper/2605.00070/) (2026)
- [FiGuRO: Intrinsic Dimension Estimation for Multi-Modal Data](https://mlsanity.com/paper/2608.10857/) (2026)

## Abstract (arXiv)

The choice of an appropriate bottleneck dimension and the application of effective regularization are both essential for Autoencoders to learn meaningful representations from unlabeled data. In this paper, we introduce a new class of deterministic autoencoders, Rank Reduction Autoencoders (RRAEs), which regularize their latent spaces by employing a truncated singular value decomposition (SVD) during training. In RRAEs, the bottleneck is defined by the rank of the latent matrix, thereby alleviating the dependence of the encoder/decoder architecture on the bottleneck size. This approach enabled us to propose an adaptive algorithm (aRRAEs) that efficiently determines the optimal bottleneck size during training. We empirically demonstrate that both RRAEs and aRRAEs are stable, scalable, and reliable, as they do not introduce any additional training hyperparameters. We evaluate our proposed architecture on a synthetic data set, as well as on MNIST, Fashion MNIST, and CelebA. Our results show that RRAEs offer several advantages over Vanilla AEs with both large and small latent spaces, and outperform other regularizing AE architectures.