# ARD-VAE: A Statistical Formulation to Find the Relevant Latent Dimensions of Variational Autoencoders

arXiv: https://arxiv.org/abs/2501.10901 · HTML: https://mlsanity.com/paper/2501.10901/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 53 references that are in the map.
- First posted on arXiv 18 Jan 2025.

## Builds on (most cited first, 26 of 26)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Automatic chemical design using a data-driven continuous representation of molecules](https://mlsanity.com/paper/1610.02415/) (2016)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [Assessing Generative Models via Precision and Recall](https://mlsanity.com/paper/1806.00035/) (2018)
- [Wasserstein Auto-Encoders](https://mlsanity.com/paper/1711.01558/) (2017)
- [Adversarial Autoencoders](https://mlsanity.com/paper/1511.05644/) (2015)
- [Variational Inference of Disentangled Latent Concepts from Unlabeled Observations](https://mlsanity.com/paper/1711.00848/) (2017)
- [Generalized Zero- and Few-Shot Learning via Aligned Variational Autoencoders](https://mlsanity.com/paper/1812.01784/) (2018)
- [Diagnosing and Enhancing VAE Models](https://mlsanity.com/paper/1903.05789/) (2019)
- [Isolating Sources of Disentanglement in Variational Autoencoders](https://mlsanity.com/paper/1802.04942/) (2018)
- [From Variational to Deterministic Autoencoders](https://mlsanity.com/paper/1903.12436/) (2019)
- [Don't Blame the ELBO! A Linear VAE Perspective on Posterior Collapse](https://mlsanity.com/paper/1911.02469/) (2019)
- [Distribution Matching in Variational Inference](https://mlsanity.com/paper/1802.06847/) (2018)
- [Few-Shot Object Detection via Variational Feature Aggregation](https://mlsanity.com/paper/2301.13411/) (2023)
- [Disentangled Variational Auto-Encoder for Semi-supervised Learning](https://mlsanity.com/paper/1709.05047/) (2017)
- [NestedVAE: Isolating Common Factors via Weak Supervision](https://mlsanity.com/paper/2002.11576/) (2020)
- [L0-ARM: Network Sparsification via Stochastic Binary Optimization](https://mlsanity.com/paper/1904.04432/) (2019)
- [GENs: Generative Encoding Networks](https://mlsanity.com/paper/2010.15283/) (2020)
- [Matching aggregate posteriors in the variational autoencoder](https://mlsanity.com/paper/2311.07693/) (2023)
- [Dynamic Narrowing of VAE Bottlenecks Using GECO and L0 Regularization](https://mlsanity.com/paper/2003.10901/) (2020)

## Built on by (most cited first, 6 of 6)

- [Adversarial Disentanglement by Backpropagation with Physics-Informed Variational Autoencoder](https://mlsanity.com/paper/2506.13658/) (2025)
- [PCAE: Learning Ordered Representations in Latent Space for Intrinsic Dimension Estimation via Principal Component Autoencoder](https://mlsanity.com/paper/2601.19179/) (2026)
- [AdaSemSeg: An Adaptive Few-shot Semantic Segmentation of Seismic Facies](https://mlsanity.com/paper/2501.16760/) (2025)
- [VSCOUT: A Hybrid Variational Autoencoder Approach to Outlier Detection in High-Dimensional Retrospective Monitoring](https://mlsanity.com/paper/2601.20830/) (2026)
- [MorphoFlow: Sparse-Supervised Generative Shape Modeling with Adaptive Latent Relevance](https://mlsanity.com/paper/2604.11636/) (2026)
- [FiGuRO: Intrinsic Dimension Estimation for Multi-Modal Data](https://mlsanity.com/paper/2608.10857/) (2026)

## Abstract (arXiv)

The variational autoencoder (VAE) is a popular, deep, latent-variable model (DLVM) due to its simple yet effective formulation for modeling the data distribution. Moreover, optimizing the VAE objective function is more manageable than other DLVMs. The bottleneck dimension of the VAE is a crucial design choice, and it has strong ramifications for the model's performance, such as finding the hidden explanatory factors of a dataset using the representations learned by the VAE. However, the size of the latent dimension of the VAE is often treated as a hyperparameter estimated empirically through trial and error. To this end, we propose a statistical formulation to discover the relevant latent factors required for modeling a dataset. In this work, we use a hierarchical prior in the latent space that estimates the variance of the latent axes using the encoded data, which identifies the relevant latent dimensions. For this, we replace the fixed prior in the VAE objective function with a hierarchical prior, keeping the remainder of the formulation unchanged. We call the proposed method the automatic relevancy detection in the variational autoencoder (ARD-VAE). We demonstrate the efficacy of the ARD-VAE on multiple benchmark datasets in finding the relevant latent dimensions and their effect on different evaluation metrics, such as FID score and disentanglement analysis.