# Variational Rank Reduction Autoencoders

arXiv: https://arxiv.org/abs/2505.09458 · HTML: https://mlsanity.com/paper/2505.09458/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 31 references that are in the map.
- First posted on arXiv 14 May 2025.

## Builds on (most cited first, 12 of 12)

- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Tutorial on Variational Autoencoders](https://mlsanity.com/paper/1606.05908/) (2016)
- [Wasserstein Auto-Encoders](https://mlsanity.com/paper/1711.01558/) (2017)
- [Equinox: neural networks in JAX via callable PyTrees and filtered transformations](https://mlsanity.com/paper/2111.00254/) (2021)
- [Lagging Inference Networks and Posterior Collapse in Variational Autoencoders](https://mlsanity.com/paper/1901.05534/) (2019)
- [Deep Generative Models in Engineering Design: A Review](https://mlsanity.com/paper/2110.10863/) (2021)
- [Explicitly Minimizing the Blur Error of Variational Autoencoders](https://mlsanity.com/paper/2304.05939/) (2023)
- [GLSR-VAE: Geodesic Latent Space Regularization for Variational AutoEncoder Architectures](https://mlsanity.com/paper/1707.04588/) (2017)
- [Preventing Posterior Collapse with Levenshtein Variational Autoencoder](https://mlsanity.com/paper/2004.14758/) (2020)
- [Beyond Vanilla Variational Autoencoders: Detecting Posterior Collapse in Conditional and Hierarchical Variational Autoencoders](https://mlsanity.com/paper/2306.05023/) (2023)
- [Adversarial Training of Variational Auto-encoders for High Fidelity Image Generation](https://mlsanity.com/paper/1804.10323/) (2018)
- [Short-Time Fourier Transform for deblurring Variational Autoencoders](https://mlsanity.com/paper/2401.03166/) (2024)

## Built on by (most cited first, 2 of 2)

- [Variational Rank Reduction Autoencoders for Generative Thermal Design](https://mlsanity.com/paper/2509.08515/) (2025)
- [Generative Parametric Design (GPD): A framework for real-time geometry generation and on-the-fly multiparametric approximation](https://mlsanity.com/paper/2512.11748/) (2025)

## Abstract (arXiv)

Deterministic Rank Reduction Autoencoders (RRAEs) enforce by construction a regularization on the latent space by applying a truncated SVD. While this regularization makes Autoencoders more powerful, using them for generative purposes is counter-intuitive due to their deterministic nature. On the other hand, Variational Autoencoders (VAEs) are well known for their generative abilities by learning a probabilistic latent space. In this paper, we present Variational Rank Reduction Autoencoders (VRRAEs), a model that leverages the advantages of both RRAEs and VAEs. Our claims and results show that when carefully sampling the latent space of RRAEs and further regularizing with the Kullback-Leibler (KL) divergence (similarly to VAEs), VRRAEs outperform RRAEs and VAEs. Additionally, we show that the regularization induced by the SVD not only makes VRRAEs better generators than VAEs, but also reduces the possibility of posterior collapse. Our results include a synthetic dataset of a small size that showcases the robustness of VRRAEs against collapse, and three real-world datasets; the MNIST, CelebA, and CIFAR-10, over which VRRAEs are shown to outperform both VAEs and RRAEs on many random generation and interpolation tasks based on the FID score. We developed an open-source implementation of VRRAEs in JAX (Equinox), available at https://github.com/JadM133/RRAEs.git.