# Mutual Information Collapse Explains Disentanglement Failure in β-VAEs

arXiv: https://arxiv.org/abs/2602.09277 · HTML: https://mlsanity.com/paper/2602.09277/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 35 references that are in the map.
- First posted on arXiv 9 Feb 2026.

## Builds on (most cited first, 19 of 19)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [Variational Inference of Disentangled Latent Concepts from Unlabeled Observations](https://mlsanity.com/paper/1711.00848/) (2017)
- [Towards Causal Representation Learning](https://mlsanity.com/paper/2102.11107/) (2021)
- [On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Dataset](https://mlsanity.com/paper/1906.03292/) (2019)
- [Measuring Disentanglement: A Review of Metrics](https://mlsanity.com/paper/2012.09276/) (2020)
- [SQ-VAE: Variational Bayes on Discrete Representation with Self-annealed Stochastic Quantization](https://mlsanity.com/paper/2205.07547/) (2022)
- [Visual Representation Learning Does Not Generalize Strongly Within the Same Domain](https://mlsanity.com/paper/2107.08221/) (2021)
- [Posterior Collapse of a Linear Latent Variable Model](https://mlsanity.com/paper/2205.04009/) (2022)
- [Scalarizing Functions in Bayesian Multiobjective Optimization](https://mlsanity.com/paper/1904.05760/) (2019)
- [Preventing Oversmoothing in VAE via Generalized Variance Parameterization](https://mlsanity.com/paper/2102.08663/) (2021)
- [Hyperbolic VAE via Latent Gaussian Distributions](https://mlsanity.com/paper/2209.15217/) (2022)
- [Learning Dynamics in Linear VAE: Posterior Collapse Threshold, Superfluous Latent Space Pitfalls, and Speedup with KL Annealing](https://mlsanity.com/paper/2310.15440/) (2023)
- [Beta-VAE Reproducibility: Challenges and Extensions](https://mlsanity.com/paper/2112.14278/) (2021)
- [High-dimensional Asymptotics of VAEs: Threshold of Posterior Collapse and Dataset-Size Dependence of Rate-Distortion Curve](https://mlsanity.com/paper/2309.07663/) (2023)
- [A Preliminary Study of Disentanglement With Insights on the Inadequacy of Metrics](https://mlsanity.com/paper/1911.11791/) (2019)
- [Compressing Biology: Evaluating the Stable Diffusion VAE for Phenotypic Drug Discovery](https://mlsanity.com/paper/2510.19887/) (2025)

## Abstract (arXiv)

The β-VAE is a foundational framework for unsupervised disentanglement, using β to regulate the trade-off between latent factorization and reconstruction fidelity. Empirically, however, disentanglement performance exhibits a pervasive non-monotonic trend: benchmarks such as MIG and SAP typically peak at intermediate β and collapse as regularization increases. We demonstrate that this collapse is a fundamental information-theoretic failure, where strong Kullback-Leibler pressure promotes marginal independence at the expense of the latent channel's semantic informativeness. By formalizing this mechanism in a linear-Gaussian setting, we prove that for β > 1, stationarity-induced dynamics trigger a spectral contraction of the encoder gain, driving latent-factor mutual information to zero. To resolve this, we introduce the λβ-VAE, which decouples regularization pressure from informational collapse via an auxiliary L2 reconstruction penalty λ. Extensive experiments on dSprites, Shapes3D, and MPI3D-real confirm that λ > 0 stabilizes disentanglement and restores latent informativeness over a significantly broader range of β, providing a principled theoretical justification for dual-parameter regularization in variational inference backbones.