# Variational Inference of Disentangled Latent Concepts from Unlabeled Observations

arXiv: https://arxiv.org/abs/1711.00848 · HTML: https://mlsanity.com/paper/1711.00848/

## Key facts

- Cited by 241 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Nov 2017.

## Built on by (most cited first, 30 of 241)

- [Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style](https://mlsanity.com/paper/2106.04619/) (2021)
- [D'ya like DAGs? A Survey on Structure Learning and Causal Discovery](https://mlsanity.com/paper/2103.02582/) (2021)
- [Illiterate DALL-E Learns to Compose](https://mlsanity.com/paper/2110.11405/) (2021)
- [CITRIS: Causal Identifiability from Temporal Intervened Sequences](https://mlsanity.com/paper/2202.03169/) (2022)
- [Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks](https://mlsanity.com/paper/2207.13243/) (2022)
- [Disentangled Representation Learning](https://mlsanity.com/paper/2211.11695/) (2022)
- [Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Task](https://mlsanity.com/paper/2310.09336/) (2023)
- [Object-Centric Slot Diffusion](https://mlsanity.com/paper/2303.10834/) (2023)
- [Generalization and Robustness Implications in Object-Centric Learning](https://mlsanity.com/paper/2107.00637/) (2021)
- [SODA: Bottleneck Diffusion Models for Representation Learning](https://mlsanity.com/paper/2311.17901/) (2023)
- [Desiderata for Representation Learning: A Causal Perspective](https://mlsanity.com/paper/2109.03795/) (2021)
- [GlanceNets: Interpretabile, Leak-proof Concept-based Models](https://mlsanity.com/paper/2205.15612/) (2022)
- [Visual Representation Learning Does Not Generalize Strongly Within the Same Domain](https://mlsanity.com/paper/2107.08221/) (2021)
- [Factorizing Knowledge in Neural Networks](https://mlsanity.com/paper/2207.03337/) (2022)
- [Function Classes for Identifiable Nonlinear Independent Component Analysis](https://mlsanity.com/paper/2208.06406/) (2022)
- [Causal Representation Learning for Instantaneous and Temporal Effects in Interactive Systems](https://mlsanity.com/paper/2206.06169/) (2022)
- [Score-based Causal Representation Learning with Interventions](https://mlsanity.com/paper/2301.08230/) (2023)
- [Nonlinear Independent Component Analysis for Principled Disentanglement in Unsupervised Deep Learning](https://mlsanity.com/paper/2303.16535/) (2023)
- [Discover the Unknown Biased Attribute of an Image Classifier](https://mlsanity.com/paper/2104.14556/) (2021)
- [Learning Disentangled Representations in the Imaging Domain](https://mlsanity.com/paper/2108.12043/) (2021)
- [Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learning](https://mlsanity.com/paper/2211.14666/) (2022)
- [Compositional Generalization in Unsupervised Compositional Representation Learning: A Study on Disentanglement and Emergent Language](https://mlsanity.com/paper/2210.00482/) (2022)
- [Disentanglement via Latent Quantization](https://mlsanity.com/paper/2305.18378/) (2023)
- [DISSECT: Disentangled Simultaneous Explanations via Concept Traversals](https://mlsanity.com/paper/2105.15164/) (2021)
- [Disentanglement of Correlated Factors via Hausdorff Factorized Support](https://mlsanity.com/paper/2210.07347/) (2022)
- [Neural Systematic Binder](https://mlsanity.com/paper/2211.01177/) (2022)
- [Subspace Identification for Multi-Source Domain Adaptation](https://mlsanity.com/paper/2310.04723/) (2023)
- [Orthogonal Jacobian Regularization for Unsupervised Disentanglement in Image Generation](https://mlsanity.com/paper/2108.07668/) (2021)
- [Emergence of Hidden Capabilities: Exploring Learning Dynamics in Concept Space](https://mlsanity.com/paper/2406.19370/) (2024)
- [Learning Disentangled Representation by Exploiting Pretrained Generative Models: A Contrastive Learning View](https://mlsanity.com/paper/2102.10543/) (2021)

## Abstract (arXiv)

Disentangled representations, where the higher level data generative factors are reflected in disjoint latent dimensions, offer several benefits such as ease of deriving invariant representations, transferability to other tasks, interpretability, etc. We consider the problem of unsupervised learning of disentangled representations from large pool of unlabeled observations, and propose a variational inference based approach to infer disentangled latent factors. We introduce a regularizer on the expectation of the approximate posterior over observed data that encourages the disentanglement. We also propose a new disentanglement metric which is better aligned with the qualitative disentanglement observed in the decoder's output. We empirically observe significant improvement over existing methods in terms of both disentanglement and data likelihood (reconstruction quality).