mlsanityData to 7 Oct 2026

Rethinking Disentanglement under Dependent Factors of Variation

arXiv 2408.07016 ↗First posted 13 Aug 2024cs.LG cs.AI stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

Representation learning is an approach that allows to discover and extract the factors of variation from the data. Intuitively, a representation is said to be disentangled if it separates the different factors of variation in a way that is understandable to humans. Definitions of disentanglement and metrics to measure it usually assume that the factors of variation are independent of each other. However, this is generally false in the real world, which limits the use of these definitions and metrics to very specific and unrealistic scenarios. In this paper we give a definition of disentanglement based on information theory that is also valid when the factors of variation are not independent. Furthermore, we relate this definition to the Information Bottleneck Method. Finally, we propose a method to measure the degree of disentanglement from the given definition that works when the factors of variation are not independent. We show through different experiments that the method proposed in this paper correctly measures disentanglement with non-independent factors of variation, while other methods fail in this scenario.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 46 of its 72 references that are in the map. First posted on arXiv 13 Aug 2024.

Builds on

46 of 72 references.

References · 46

  1. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  2. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
  3. 2016Deep Variational Information Bottleneckcited by 915
  4. 2018Multimodal Unsupervised Image-to-Image Translationcited by 795
  5. 2018Disentangling by Factorisingcited by 614
  6. 2018Explaining Explanations: An Overview of Interpretability of Machine Learningcited by 508
  7. 2017Opening the Black Box of Deep Neural Networks via Informationcited by 494
  8. 2018Understanding disentangling in β-VAEcited by 332
  9. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 299
  10. 2017Variational Inference of Disentangled Latent Concepts from Unlabeled Observationscited by 240
  11. 2020Weakly-Supervised Disentanglement Without Compromisescited by 237
  12. 2017Emergence of Invariance and Disentanglement in Deep Representationscited by 213
34 more
  1. 2018Visual Interpretability for Deep Learning: a Surveycited by 155
  2. 2020A Theory of Usable Information Under Computational Constraintscited by 153
  3. 2019On the Fairness of Disentangled Representationscited by 144
  4. 2022Interventional Causal Representation Learningcited by 140
  5. 2016Information Dropout: Learning Optimal Representations Through Noisy Computationcited by 131
  6. 2015Deep Convolutional Inverse Graphics Networkcited by 121
  7. 2019Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEscited by 116
  8. 2019Towards Visually Explaining Variational Autoencoderscited by 113
  9. 2018Robustly Disentangled Causal Mechanisms: Validating Deep Representations for Interventional Robustnesscited by 108
  10. 2023Nonparametric Identifiability of Causal Representations from Unknown Interventionscited by 105
  11. 2018Learning Deep Disentangled Embeddings with the F-Statistic Losscited by 101
  12. 2019On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Datasetcited by 100
  13. 2020On Disentangled Representations Learned From Correlated Datacited by 94
  14. 2015Attribute2Image: Conditional Image Generation from Visual Attributescited by 93
  15. 2017Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Datacited by 92
  16. 2018Learning Disentangled Joint Continuous and Discrete Representationscited by 83
  17. 2020Fairness by Learning Orthogonal Disentangled Representationscited by 76
  18. 2017Learning Disentangled Representations with Semi-Supervised Deep Generative Modelscited by 72
  19. 2020Measuring Disentanglement: A Review of Metricscited by 69
  20. 2020Weakly Supervised Disentangled Generative Causal Representation Learningcited by 67
  21. 2019Theory and Evaluation Metrics for Learning Disentangled Representationscited by 57
  22. 2014Learning the Irreducible Representations of Commutative Lie Groupscited by 41
  23. 2022Disentanglement of Correlated Factors via Hausdorff Factorized Supportcited by 37
  24. 2017Interpretable Transformations with Encoder-Decoder Networkscited by 33
  25. 2018Style and Content Disentanglement in Generative Adversarial Networkscited by 29
  26. 2021Where and What? Examining Interpretable Disentangled Representationscited by 27
  27. 2016A Survey of Inductive Biases for Factorial Representation-Learningcited by 23
  28. 2019Relevance Factor VAE: Learning and Identifying Disentangled Factorscited by 20
  29. 2020Evaluating the Disentanglement of Deep Generative Models through Manifold Topologycited by 20
  30. 2019Disentangling and Learning Robust Representations with Natural Clusteringcited by 10
  31. 2020Discond-VAE: Disentangling Continuous Factors from the Discretecited by 4
  32. 2019How to Not Measure Disentanglementcited by 2
  33. 2024Unsupervised Multiple Domain Translation through Controlled Disentanglement in Variational Autoencodercited by 2
  34. 2024Predefined Prototypes for Intra-Class Separation and Disentanglementcited by 2

Built on by

Not cited yet.