# Variantional autoencoder with decremental information bottleneck for disentanglement

arXiv: https://arxiv.org/abs/2303.12959 · HTML: https://mlsanity.com/paper/2303.12959/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 50 references that are in the map.
- First posted on arXiv 22 Mar 2023.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [Improving Variational Inference with Inverse Autoregressive Flow](https://mlsanity.com/paper/1606.04934/) (2016)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [Understanding disentangling in β-VAE](https://mlsanity.com/paper/1804.03599/) (2018)
- [Ladder Variational Autoencoders](https://mlsanity.com/paper/1602.02282/) (2016)
- [Towards a Definition of Disentangled Representations](https://mlsanity.com/paper/1812.02230/) (2018)
- [Variational Inference of Disentangled Latent Concepts from Unlabeled Observations](https://mlsanity.com/paper/1711.00848/) (2017)
- [On Disentangled Representations Learned From Correlated Data](https://mlsanity.com/paper/2006.07886/) (2020)
- [Learning Disentangled Joint Continuous and Discrete Representations](https://mlsanity.com/paper/1804.00104/) (2018)
- [On the Transfer of Disentangled Representations in Realistic Settings](https://mlsanity.com/paper/2010.14407/) (2020)
- [Structured Disentangled Representations](https://mlsanity.com/paper/1804.02086/) (2018)
- [Theory and Evaluation Metrics for Learning Disentangled Representations](https://mlsanity.com/paper/1908.09961/) (2019)
- [Unsupervised Model Selection for Variational Disentangled Representation Learning](https://mlsanity.com/paper/1905.12614/) (2019)
- [Orthogonal Jacobian Regularization for Unsupervised Disentanglement in Image Generation](https://mlsanity.com/paper/2108.07668/) (2021)
- [Learning Discrete and Continuous Factors of Data via Alternating Disentanglement](https://mlsanity.com/paper/1905.09432/) (2019)
- [An Empirical Study on Disentanglement of Negative-free Contrastive Learning](https://mlsanity.com/paper/2206.04756/) (2022)
- [Siamese Prototypical Contrastive Learning](https://mlsanity.com/paper/2208.08819/) (2022)
- [Rethinking Prototypical Contrastive Learning through Alignment, Uniformity and Correlation](https://mlsanity.com/paper/2210.10194/) (2022)
- [A Preliminary Study of Disentanglement With Insights on the Inadequacy of Metrics](https://mlsanity.com/paper/1911.11791/) (2019)

## Built on by (most cited first, 1 of 1)

- [Learning Group Actions In Disentangled Latent Image Representations](https://mlsanity.com/paper/2512.04015/) (2025)

## Abstract (arXiv)

One major challenge of disentanglement learning with variational autoencoders is the trade-off between disentanglement and reconstruction fidelity. Previous studies, which increase the information bottleneck during training, tend to lose the constraint of disentanglement, leading to the information diffusion problem. In this paper, we present a novel framework for disentangled representation learning, DeVAE, which utilizes hierarchical latent spaces with decreasing information bottlenecks across these spaces. The key innovation of our approach lies in connecting the hierarchical latent spaces through disentanglement-invariant transformations, allowing the sharing of disentanglement properties among spaces while maintaining an acceptable level of reconstruction performance. We demonstrate the effectiveness of DeVAE in achieving a balance between disentanglement and reconstruction through a series of experiments and ablation studies on dSprites and Shapes3D datasets. Code is available at https://github.com/erow/disentanglement_lib/tree/pytorch#devae.