# Beta-VAE Reproducibility: Challenges and Extensions

arXiv: https://arxiv.org/abs/2112.14278 · HTML: https://mlsanity.com/paper/2112.14278/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 27 references that are in the map.
- First posted on arXiv 28 Dec 2021.

## Builds on (most cited first, 14 of 14)

- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [Adversarial Autoencoders](https://mlsanity.com/paper/1511.05644/) (2015)
- [Understanding disentangling in β-VAE](https://mlsanity.com/paper/1804.03599/) (2018)
- [Are GANs Created Equal? A Large-Scale Study](https://mlsanity.com/paper/1711.10337/) (2017)
- [Diagnosing and Enhancing VAE Models](https://mlsanity.com/paper/1903.05789/) (2019)
- [On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Dataset](https://mlsanity.com/paper/1906.03292/) (2019)
- [How good is my GAN?](https://mlsanity.com/paper/1807.09499/) (2018)
- [Dual Contradistinctive Generative Autoencoder](https://mlsanity.com/paper/2011.10063/) (2020)
- [Simple and Effective VAE Training with Calibrated Decoders](https://mlsanity.com/paper/2006.13202/) (2020)
- [A Sober Look at the Unsupervised Learning of Disentangled Representations and their Evaluation](https://mlsanity.com/paper/2010.14766/) (2020)

## Built on by (most cited first, 10 of 10)

- [Neuromorphic Visual Scene Understanding with Resonator Networks](https://mlsanity.com/paper/2208.12880/) (2022)
- [DySurv: dynamic deep learning model for survival analysis with conditional variational inference](https://mlsanity.com/paper/2310.18681/) (2023)
- [Learning Metal Microstructural Heterogeneity through Spatial Mapping of Diffraction Latent Space Features](https://mlsanity.com/paper/2501.18064/) (2025)
- [Gamma-VAE: Curvature regularized variational autoencoders for uncovering emergent low dimensional geometric structure in high dimensional data](https://mlsanity.com/paper/2403.01078/) (2024)
- [L-VAE: Variational Auto-Encoder with Learnable Beta for Disentangled Representation](https://mlsanity.com/paper/2507.02619/) (2025)
- [Mutual Information Collapse Explains Disentanglement Failure in β-VAEs](https://mlsanity.com/paper/2602.09277/) (2026)
- [Generalized multi-object classification and tracking with sparse feature resonator networks](https://mlsanity.com/paper/2603.22539/) (2026)
- [Orthogonal Dendritic Intrinsic Networks: An Architecture for Significance-Ordered, Orthogonal Latent Spaces](https://mlsanity.com/paper/2607.05653/) (2026)
- [A Systematic Evaluation of Molecule Generation Models for De Novo Drug Design: From Benchmarks to Practical Insights](https://mlsanity.com/paper/2609.10099/) (2026)
- [Evaluating Physical Consistency and Plausibility in Generative Scenario Models for Autonomous Driving](https://mlsanity.com/paper/2610.01581/) (2026)

## Abstract (arXiv)

β-VAE is a follow-up technique to variational autoencoders that proposes special weighting of the KL divergence term in the VAE loss to obtain disentangled representations. Unsupervised learning is known to be brittle even on toy datasets and a meaningful, mathematically precise definition of disentanglement remains difficult to find. Here we investigate the original β-VAE paper and add evidence to the results previously obtained indicating its lack of reproducibility. We also further expand the experimentation of the models and include further more complex datasets in the analysis. We also implement an FID scoring metric for the β-VAE model and conclude a qualitative analysis of the results obtained. We end with a brief discussion on possible future investigations that can be conducted to add more robustness to the claims.