Enriching Disentanglement: From Logical Definitions to Quantitative Metrics
Not on the map. Why?
Abstract (arXiv)
Disentangling the explanatory factors in complex data is a promising approach for generalizable and data-efficient representation learning. While a variety of quantitative metrics for learning and evaluating disentangled representations have been proposed, it remains unclear what properties these metrics truly quantify. In this work, we establish algebraic relationships between logical definitions and quantitative metrics to derive theoretically grounded disentanglement metrics. Concretely, we introduce a compositional approach for converting a higher-order predicate into a real-valued quantity by replacing (i) equality with a strict premetric, (ii) the Heyting algebra of binary truth values with a quantale of continuous values, and (iii) quantifiers with aggregators. The metrics induced by logical definitions have strong theoretical guarantees, and some of them are easily differentiable and can be used as learning objectives directly. Finally, we empirically demonstrate the effectiveness of the proposed metrics by isolating different aspects of disentangled representations.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 62 of its 125 references that are in the map. First posted on arXiv 19 May 2023.
Builds on
62 of 125 references.
References · 62
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2015Variational Inference with Normalizing Flowscited by 2,161
- 2017Deep Setscited by 1,394
- 2016Group Equivariant Convolutional Networkscited by 1,154
- 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
- 2018Disentangling by Factorisingcited by 614
- 2020A Survey on Contrastive Self-supervised Learningcited by 553
- 2016Input Convex Neural Networkscited by 407
- 2018Invertible Residual Networkscited by 364
- 2018Invariant and Equivariant Graph Networkscited by 339
- 2020A Metric Learning Reality Checkcited by 317
- 2019Robust Aggregation for Federated Learningcited by 314
38 more
- 2018Towards a Definition of Disentangled Representationscited by 294
- 2020Equivariant Flows: Exact Likelihood Generative Learning for Symmetric Densitiescited by 248
- 2020Weakly-Supervised Disentanglement Without Compromisescited by 237
- 2019On the Fairness of Disentangled Representationscited by 144
- 2022Weakly supervised causal representation learningcited by 140
- 2018Robustly Disentangled Causal Mechanisms: Validating Deep Representations for Interventional Robustnesscited by 108
- 2023Equivariant Architectures for Learning in Deep Weight Spacescited by 106
- 2018Learning Deep Disentangled Embeddings with the F-Statistic Losscited by 101
- 2019On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Datasetcited by 100
- 2020On Disentangled Representations Learned From Correlated Datacited by 94
- 2019Weakly Supervised Disentanglement with Guaranteescited by 82
- 2020Measuring Disentanglement: A Review of Metricscited by 69
- 2020On the Transfer of Disentangled Representations in Realistic Settingscited by 63
- 2019Theory and Evaluation Metrics for Learning Disentangled Representationscited by 57
- 2019A synthetic approach to Markov kernels, conditional independence and theorems on sufficient statisticscited by 55
- 2022Graph Neural Networks are Dynamic Programmerscited by 55
- 2021Categorical Foundations of Gradient-Based Learningcited by 44
- 2022Compositional Generalization in Unsupervised Compositional Representation Learning: A Study on Disentanglement and Emergent Languagecited by 42
- 2014Learning the Irreducible Representations of Commutative Lie Groupscited by 41
- 2019On the Calibration of Multiclass Classification with Rejectioncited by 41
- 2022From Statistical to Causal Learningcited by 40
- 2016Geometric Median in Nearly Linear Timecited by 38
- 2022Disentanglement of Correlated Factors via Hausdorff Factorized Supportcited by 37
- 2022On the Power of Foundation Modelscited by 37
- 2020Calibrated Surrogate Losses for Adversarially Robust Classificationcited by 36
- 2021Category Theory in Machine Learningcited by 36
- 2019Symmetry-Based Disentangled Representation Learning requires Interaction with Environmentscited by 35
- 2020Natural Graph Networkscited by 35
- 2020Progressive Learning and Disentanglement of Hierarchical Representationscited by 30
- 2022Universal approximation property of invertible neural networkscited by 30
- 2020Disentangling by Subspace Diffusioncited by 28
- 2021Multi-Agent MDP Homomorphic Networkscited by 24
- 2022DCI-ES: An Extended Disentanglement Framework with Connections to Identifiabilitycited by 23
- 2021Towards Building A Group-based Unsupervised Representation Disentanglement Frameworkcited by 22
- 2021An enriched category theory of language: from syntax to semanticscited by 21
- 2020Evaluating the Disentanglement of Deep Generative Models through Manifold Topologycited by 20
- 2021Learning disentangled representations via product manifold projectioncited by 20
- 2014Transformation Properties of Learned Visual Representationscited by 18
50 most cited of 62 All 62 (JSON)
Built on by
Citing papers per year: 2024: 1; 2025: 0; 2026: 1.
2 papers.