# Partial Disentanglement via Mechanism Sparsity

arXiv: https://arxiv.org/abs/2207.07732 · HTML: https://mlsanity.com/paper/2207.07732/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 35 references that are in the map.
- First posted on arXiv 15 Jul 2022.

## Builds on (most cited first, 18 of 18)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Variational Autoencoders and Nonlinear ICA: A Unifying Framework](https://mlsanity.com/paper/1907.04809/) (2019)
- [Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style](https://mlsanity.com/paper/2106.04619/) (2021)
- [Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning](https://mlsanity.com/paper/1805.08651/) (2018)
- [Inductive Biases for Deep Learning of Higher-Level Cognition](https://mlsanity.com/paper/2011.15091/) (2020)
- [Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA](https://mlsanity.com/paper/1605.06336/) (2016)
- [Weakly-Supervised Disentanglement Without Compromises](https://mlsanity.com/paper/2002.02886/) (2020)
- [Towards Causal Representation Learning](https://mlsanity.com/paper/2102.11107/) (2021)
- [Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICA](https://mlsanity.com/paper/2107.10098/) (2021)
- [CITRIS: Causal Identifiability from Temporal Intervened Sequences](https://mlsanity.com/paper/2202.03169/) (2022)
- [ICE-BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICA](https://mlsanity.com/paper/2002.11537/) (2020)
- [Learning Temporally Causal Latent Processes from General Temporal Data](https://mlsanity.com/paper/2110.05428/) (2021)
- [Weakly Supervised Representation Learning with Sparse Perturbations](https://mlsanity.com/paper/2206.01101/) (2022)
- [Unsupervised Model Selection for Variational Disentangled Representation Learning](https://mlsanity.com/paper/1905.12614/) (2019)
- [Properties from Mechanisms: An Equivariance Perspective on Identifiable Representation Learning](https://mlsanity.com/paper/2110.15796/) (2021)

## Built on by (most cited first, 29 of 29)

- [Nonparametric Identifiability of Causal Representations from Unknown Interventions](https://mlsanity.com/paper/2306.00542/) (2023)
- [Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolation](https://mlsanity.com/paper/2307.02598/) (2023)
- [Causal Component Analysis](https://mlsanity.com/paper/2305.17225/) (2023)
- [Provably Learning Object-Centric Representations](https://mlsanity.com/paper/2305.14229/) (2023)
- [Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learning](https://mlsanity.com/paper/2211.14666/) (2022)
- [BISCUIT: Causal Representation Learning from Binary Interactions](https://mlsanity.com/paper/2306.09643/) (2023)
- [Learning Causal Representations of Single Cells via Sparse Mechanism Shift Modeling](https://mlsanity.com/paper/2211.03553/) (2022)
- [Generalizing Nonlinear ICA Beyond Structural Sparsity](https://mlsanity.com/paper/2311.00866/) (2023)
- [Multi-Domain Causal Representation Learning via Weak Distributional Invariances](https://mlsanity.com/paper/2310.02854/) (2023)
- [Marrying Causal Representation Learning with Dynamical Systems for Science](https://mlsanity.com/paper/2405.13888/) (2024)
- [Object-centric architectures enable efficient causal representation learning](https://mlsanity.com/paper/2310.19054/) (2023)
- [Identifying Linearly-Mixed Causal Representations from Multi-Node Interventions](https://mlsanity.com/paper/2311.02695/) (2023)
- [On the Identification of Temporally Causal Representation with Instantaneous Dependence](https://mlsanity.com/paper/2405.15325/) (2024)
- [Identifiable Causal Representation Learning: Unsupervised, Multi-View, and Multi-Environment](https://mlsanity.com/paper/2406.13371/) (2024)
- [Cooper: A Library for Constrained Optimization in Deep Learning](https://mlsanity.com/paper/2504.01212/) (2025)
- [Language Agents Meet Causality -- Bridging LLMs and Causal World Models](https://mlsanity.com/paper/2410.19923/) (2024)
- [Debiasing Reward Models by Representation Learning with Guarantees](https://mlsanity.com/paper/2510.23751/) (2025)
- [Sanity Checking Causal Representation Learning on a Simple Real-World System](https://mlsanity.com/paper/2502.20099/) (2025)
- [Toward the Identifiability of Comparative Deep Generative Models](https://mlsanity.com/paper/2401.15903/) (2024)
- [From Orthogonality to Dependency: Learning Disentangled Representation for Multi-Modal Time-Series Sensing Signals](https://mlsanity.com/paper/2405.16083/) (2024)
- [CausalVerse: Benchmarking Causal Representation Learning with Configurable High-Fidelity Simulations](https://mlsanity.com/paper/2510.14049/) (2025)
- [Mechanistic Independence: A Principle for Identifiable Disentangled Representations](https://mlsanity.com/paper/2509.22196/) (2025)
- [Disentangling Dynamical Systems: Causal Representation Learning Meets Local Sparse Attention](https://mlsanity.com/paper/2603.14483/) (2026)
- [Supervised Contrastive Block Disentanglement](https://mlsanity.com/paper/2502.07281/) (2025)
- [Online Time Series Forecasting with Theoretical Guarantees](https://mlsanity.com/paper/2510.18281/) (2025)
- [Causal View of Time Series Imputation: Some Identification Results on Missing Mechanism](https://mlsanity.com/paper/2505.07180/) (2025)
- [Hierarchical Action Learning for Weakly-Supervised Action Segmentation](https://mlsanity.com/paper/2602.24275/) (2026)
- [Multimodal LLMs under Pairwise Modalities](https://mlsanity.com/paper/2605.21059/) (2026)
- [Hessian Rank Constraint for Learning Structure of Nonlinear Latent Variable Models](https://mlsanity.com/paper/2609.24241/) (2026)

## Abstract (arXiv)

Disentanglement via mechanism sparsity was introduced recently as a principled approach to extract latent factors without supervision when the causal graph relating them in time is sparse, and/or when actions are observed and affect them sparsely. However, this theory applies only to ground-truth graphs satisfying a specific criterion. In this work, we introduce a generalization of this theory which applies to any ground-truth graph and specifies qualitatively how disentangled the learned representation is expected to be, via a new equivalence relation over models we call consistency. This equivalence captures which factors are expected to remain entangled and which are not based on the specific form of the ground-truth graph. We call this weaker form of identifiability partial disentanglement. The graphical criterion that allows complete disentanglement, proposed in an earlier work, can be derived as a special case of our theory. Finally, we enforce graph sparsity with constrained optimization and illustrate our theory and algorithm in simulations.