mlsanityData to 7 Oct 2026

Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations

arXiv 1811.12359 ↗First posted 29 Nov 2018cs.LG cs.AI stat.MLcited by 929

Not on the map. Why?

Cited by 31 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The key idea behind the unsupervised learning of disentangled representations is that real-world data is generated by a few explanatory factors of variation which can be recovered by unsupervised learning algorithms. In this paper, we provide a sober look at recent progress in the field and challenge some common assumptions. We first theoretically show that the unsupervised learning of disentangled representations is fundamentally impossible without inductive biases on both the models and the data. Then, we train more than 12000 models covering most prominent methods and evaluation metrics in a reproducible large-scale experimental study on seven different data sets. We observe that while the different methods successfully enforce properties ``encouraged'' by the corresponding losses, well-disentangled models seemingly cannot be identified without supervision. Furthermore, increased disentanglement does not seem to lead to a decreased sample complexity of learning for downstream tasks. Our results suggest that future work on disentanglement learning should be explicit about the role of inductive biases and (implicit) supervision, investigate concrete benefits of enforcing disentanglement of the learned representations, and consider a reproducible experimental setup covering several data sets.

Key facts. Cited by 929 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Nov 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 1; 2021: 158; 2022: 174; 2023: 160; 2024: 121; 2025: 144; 2026: 170.

929 papers.

Sort:

By area:

  1. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 431
  2. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  3. 2021Self-Supervised Learning with Data Augmentations Provably Isolates Content from Stylecited by 308
  4. 2023Diffusion Self-Guidance for Controllable Image Generationcited by 305
  5. 2021Contrastive Learning Inverts the Data Generating Processcited by 220
  6. 2023Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Modelscited by 218
  7. 2021D'ya like DAGs? A Survey on Structure Learning and Causal Discoverycited by 209
  8. 2021Towards Causal Representation Learningcited by 207
  9. 2021A Fine-Grained Analysis on Distribution Shiftcited by 171
  10. 2021Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICAcited by 158
  11. 2023StyleDiffusion: Controllable Disentangled Style Transfer via Diffusion Modelscited by 154
  12. 2022Weakly supervised causal representation learningcited by 140
38 more
  1. 2022Interventional Causal Representation Learningcited by 140
  2. 2024ODIN: Disentangled Reward Mitigates Hacking in RLHFcited by 133
  3. 2022CITRIS: Causal Identifiability from Temporal Intervened Sequencescited by 132
  4. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
  5. 2021Counterfactual Generative Networkscited by 111
  6. 2022Disentangled Representation Learningcited by 111
  7. 2023Nonparametric Identifiability of Causal Representations from Unknown Interventionscited by 105
  8. 2021Independent mechanism analysis, a new concept?cited by 96
  9. 2021Learning Temporally Causal Latent Processes from General Temporal Datacited by 96
  10. 2021Counterfactual Zero-Shot and Open-Set Visual Recognitioncited by 95
  11. 2022Progressive Disentangled Representation Learning for Fine-Grained Controllable Talking Head Synthesiscited by 93
  12. 2023Learning Linear Causal Representations from Interventions under General Nonlinear Mixingcited by 92
  13. 2023Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Taskcited by 91
  14. 2022On the Identifiability of Nonlinear ICA: Sparsity and Beyondcited by 84
  15. 2022Temporal Attention Unit: Towards Efficient Spatiotemporal Predictive Learningcited by 84
  16. 2021Evading the Simplicity Bias: Training a Diverse Set of Models Discovers Solutions with Superior OOD Generalizationcited by 83
  17. 2021Size-Invariant Graph Representations for Graph Classification Extrapolationscited by 82
  18. 2021Generalization and Robustness Implications in Object-Centric Learningcited by 81
  19. 2021Explainability of deep vision-based autonomous driving systems: Review and challengescited by 80
  20. 2024RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representationscited by 77
  21. 2021Desiderata for Representation Learning: A Causal Perspectivecited by 76
  22. 2023SODA: Bottleneck Diffusion Models for Representation Learningcited by 76
  23. 2022GlanceNets: Interpretabile, Leak-proof Concept-based Modelscited by 73
  24. 2025A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Modelscited by 73
  25. 2023Multi-View Causal Representation Learning with Partial Observabilitycited by 72
  26. 2024ObjectDrop: Bootstrapping Counterfactuals for Photorealistic Object Removal and Insertioncited by 71
  27. 2021What Matters for Adversarial Imitation Learning?cited by 70
  28. 2022Identifiability of deep generative models without auxiliary informationcited by 70
  29. 2025Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Modelscited by 67
  30. 2021Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICAcited by 64
  31. 2022Weakly Supervised Representation Learning with Sparse Perturbationscited by 64
  32. 2023ITI-GEN: Inclusive Text-to-Image Generationcited by 64
  33. 2023Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcutscited by 63
  34. 2021Visual Representation Learning Does Not Generalize Strongly Within the Same Domaincited by 60
  35. 2022Factorizing Knowledge in Neural Networkscited by 59
  36. 2022Temporally Disentangled Representation Learningcited by 59
  37. 2022ZIN: When and How to Learn Invariance Without Environment Partition?cited by 58
  38. 2023Identifiability of latent-variable and structural-equation models: from linear to nonlinearcited by 58

50 most cited of 929 All 929 (JSON)