# Disentangling Latent Space for VAE by Label Relevant/Irrelevant Dimensions

arXiv: https://arxiv.org/abs/1812.09502 · HTML: https://mlsanity.com/paper/1812.09502/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Dec 2018.

## Built on by (most cited first, 19 of 19)

- [Directional Connectivity-based Segmentation of Medical Images](https://mlsanity.com/paper/2304.00145/) (2023)
- [Isometric Representation Learning for Disentangled Latent Space of Diffusion Models](https://mlsanity.com/paper/2407.11451/) (2024)
- [Lossless Compression with Probabilistic Circuits](https://mlsanity.com/paper/2111.11632/) (2021)
- [SepVAE: a contrastive VAE to separate pathological patterns from healthy ones](https://mlsanity.com/paper/2307.06206/) (2023)
- [EXoN: EXplainable encoder Network](https://mlsanity.com/paper/2105.10867/) (2021)
- [SKILL-IL: Disentangling Skill and Knowledge in Multitask Imitation Learning](https://mlsanity.com/paper/2205.03130/) (2022)
- [ID-Unet: Iterative Soft and Hard Deformation for View Synthesis](https://mlsanity.com/paper/2103.02264/) (2021)
- [Causally Disentangled Generative Variational AutoEncoder](https://mlsanity.com/paper/2302.11737/) (2023)
- [Plasma State Monitoring and Disruption Characterization using Multimodal VAEs](https://mlsanity.com/paper/2504.17710/) (2025)
- [Bridging the Gap between Label- and Reference-based Synthesis in Multi-attribute Image-to-Image Translation](https://mlsanity.com/paper/2110.05055/) (2021)
- [Self-supervised Latent Space Optimization with Nebula Variational Coding](https://mlsanity.com/paper/2506.01414/) (2025)
- [Unsupervised Synthetic Image Attribution: Alignment and Disentanglement](https://mlsanity.com/paper/2601.22663/) (2026)
- [Marginal Matching Does Not License Factorized Sampling: Auditing Conditional Style Leakage in Factorized Generative Models](https://mlsanity.com/paper/2608.05243/) (2026)
- [VAE-CE: Visual Contrastive Explanation using Disentangled VAEs](https://mlsanity.com/paper/2108.09159/) (2021)
- [LSC-GAN: Latent Style Code Modeling for Continuous Image-to-image Translation](https://mlsanity.com/paper/2110.05052/) (2021)
- [Contrastive Representation Disentanglement for Clustering](https://mlsanity.com/paper/2306.05439/) (2023)
- [Disentangling Genotype and Environment Specific Latent Features for Improved Trait Prediction using a Compositional Autoencoder](https://mlsanity.com/paper/2410.19922/) (2024)
- [Harmonizing Generalization and Personalization in Ring-topology Decentralized Federated Learning](https://mlsanity.com/paper/2504.19103/) (2025)
- [LCIP: Loss-Controlled Inverse Projection of High-Dimensional Image Data](https://mlsanity.com/paper/2602.11141/) (2026)

## Abstract (arXiv)

VAE requires the standard Gaussian distribution as a prior in the latent space. Since all codes tend to follow the same prior, it often suffers the so-called "posterior collapse". To avoid this, this paper introduces the class specific distribution for the latent code. But different from CVAE, we present a method for disentangling the latent space into the label relevant and irrelevant dimensions, bmzs and bmzu, for a single input. We apply two separated encoders to map the input into bmzs and bmzu respectively, and then give the concatenated code to the decoder to reconstruct the input. The label irrelevant code bmzu represent the common characteristics of all inputs, hence they are constrained by the standard Gaussian, and their encoder is trained in amortized variational inference way, like VAE. While bmzs is assumed to follow the Gaussian mixture distribution in which each component corresponds to a particular class. The parameters for the Gaussian components in bmzs encoder are optimized by the label supervision in a global stochastic way. In theory, we show that our method is actually equivalent to adding a KL divergence term on the joint distribution of bmzs and the class label c, and it can directly increase the mutual information between bmzs and the label c. Our model can also be extended to GAN by adding a discriminator in the pixel domain so that it produces high quality and diverse images.