# Independent Subspace Analysis for Unsupervised Learning of Disentangled Representations

arXiv: https://arxiv.org/abs/1909.05063 · HTML: https://mlsanity.com/paper/1909.05063/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Sep 2019.

## Built on by (most cited first, 8 of 8)

- [Indeterminacy in Generative Models: Characterization and Strong Identifiability](https://mlsanity.com/paper/2206.00801/) (2022)
- [Local Disentanglement in Variational Auto-Encoders Using Jacobian L1 Regularization](https://mlsanity.com/paper/2106.02923/) (2021)
- [Exploring the Latent Space of Autoencoders with Interventional Assays](https://mlsanity.com/paper/2106.16091/) (2021)
- [Controlling Text Edition by Changing Answers of Specific Questions](https://mlsanity.com/paper/2105.11018/) (2021)
- [Demystifying Inductive Biases for β-VAE Based Architectures](https://mlsanity.com/paper/2102.06822/) (2021)
- [A Revisit of Total Correlation in Disentangled Variational Auto-Encoder with Partial Disentanglement](https://mlsanity.com/paper/2502.02279/) (2025)
- [In-Context Learning Operates as Concept Subspace Learning](https://mlsanity.com/paper/2605.18830/) (2026)
- [DUET: 2D Structured and Approximately Equivariant Representations](https://mlsanity.com/paper/2306.16058/) (2023)

## Abstract (arXiv)

Recently there has been an increased interest in unsupervised learning of disentangled representations using the Variational Autoencoder (VAE) framework. Most of the existing work has focused largely on modifying the variational cost function to achieve this goal. We first show that these modifications, e.g. beta-VAE, simplify the tendency of variational inference to underfit causing pathological over-pruning and over-orthogonalization of learned components. Second we propose a complementary approach: to modify the probabilistic model with a structured latent prior. This prior allows to discover latent variable representations that are structured into a hierarchy of independent vector spaces. The proposed prior has three major advantages: First, in contrast to the standard VAE normal prior the proposed prior is not rotationally invariant. This resolves the problem of unidentifiability of the standard VAE normal prior. Second, we demonstrate that the proposed prior encourages a disentangled latent representation which facilitates learning of disentangled representations. Third, extensive quantitative experiments demonstrate that the prior significantly mitigates the trade-off between reconstruction loss and disentanglement over the state of the art.