# Weakly Supervised Disentanglement with Guarantees

arXiv: https://arxiv.org/abs/1910.09772 · HTML: https://mlsanity.com/paper/1910.09772/

## Key facts

- Cited by 82 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Oct 2019.

## Built on by (most cited first, 30 of 82)

- [Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style](https://mlsanity.com/paper/2106.04619/) (2021)
- [Towards Causal Representation Learning](https://mlsanity.com/paper/2102.11107/) (2021)
- [Weakly supervised causal representation learning](https://mlsanity.com/paper/2203.16437/) (2022)
- [Causality Inspired Representation Learning for Domain Generalization](https://mlsanity.com/paper/2203.14237/) (2022)
- [Nonparametric Identifiability of Causal Representations from Unknown Interventions](https://mlsanity.com/paper/2306.00542/) (2023)
- [Desiderata for Representation Learning: A Causal Perspective](https://mlsanity.com/paper/2109.03795/) (2021)
- [GlanceNets: Interpretabile, Leak-proof Concept-based Models](https://mlsanity.com/paper/2205.15612/) (2022)
- [Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcuts](https://mlsanity.com/paper/2305.19951/) (2023)
- [Visual Representation Learning Does Not Generalize Strongly Within the Same Domain](https://mlsanity.com/paper/2107.08221/) (2021)
- [Identifiability Results for Multimodal Contrastive Learning](https://mlsanity.com/paper/2303.09166/) (2023)
- [Score-based Causal Representation Learning with Interventions](https://mlsanity.com/paper/2301.08230/) (2023)
- [Provable Compositional Generalization for Object-Centric Learning](https://mlsanity.com/paper/2310.05327/) (2023)
- [Disentanglement via Latent Quantization](https://mlsanity.com/paper/2305.18378/) (2023)
- [DISSECT: Disentangled Simultaneous Explanations via Concept Traversals](https://mlsanity.com/paper/2105.15164/) (2021)
- [Measuring axiomatic soundness of counterfactual image models](https://mlsanity.com/paper/2303.01274/) (2023)
- [General Identifiability and Achievability for Causal Representation Learning](https://mlsanity.com/paper/2310.15450/) (2023)
- [Properties from Mechanisms: An Equivariance Perspective on Identifiable Representation Learning](https://mlsanity.com/paper/2110.15796/) (2021)
- [Conditional Mutual Information for Disentangled Representations in Reinforcement Learning](https://mlsanity.com/paper/2305.14133/) (2023)
- [Interactive Disentanglement: Learning Concepts by Interacting with their Prototype Representations](https://mlsanity.com/paper/2112.02290/) (2021)
- [DCI-ES: An Extended Disentanglement Framework with Connections to Identifiability](https://mlsanity.com/paper/2210.00364/) (2022)
- [On Causally Disentangled Representations](https://mlsanity.com/paper/2112.05746/) (2021)
- [Interpretability is in the Mind of the Beholder: A Causal Framework for Human-interpretable Representation Learning](https://mlsanity.com/paper/2309.07742/) (2023)
- [Learning disentangled representations via product manifold projection](https://mlsanity.com/paper/2103.01638/) (2021)
- [Temporal Disentanglement of Representations for Improved Generalisation in Reinforcement Learning](https://mlsanity.com/paper/2207.05480/) (2022)
- [On the Versatile Uses of Partial Distance Correlation in Deep Learning](https://mlsanity.com/paper/2207.09684/) (2022)
- [Analyzing (In)Abilities of SAEs via Formal Languages](https://mlsanity.com/paper/2410.11767/) (2024)
- [Improving Compositional Generalization Using Iterated Learning and Simplicial Embeddings](https://mlsanity.com/paper/2310.18777/) (2023)
- [Addressing the Topological Defects of Disentanglement via Distributed Operators](https://mlsanity.com/paper/2102.05623/) (2021)
- [Towards efficient representation identification in supervised learning](https://mlsanity.com/paper/2204.04606/) (2022)
- [Neural Concept Binder](https://mlsanity.com/paper/2406.09949/) (2024)

## Abstract (arXiv)

Learning disentangled representations that correspond to factors of variation in real-world data is critical to interpretable and human-controllable machine learning. Recently, concerns about the viability of learning disentangled representations in a purely unsupervised manner has spurred a shift toward the incorporation of weak supervision. However, there is currently no formalism that identifies when and how weak supervision will guarantee disentanglement. To address this issue, we provide a theoretical framework to assist in analyzing the disentanglement guarantees (or lack thereof) conferred by weak supervision when coupled with learning algorithms based on distribution matching. We empirically verify the guarantees and limitations of several weak supervision methods (restricted labeling, match-pairing, and rank-pairing), demonstrating the predictive power and usefulness of our theoretical framework.