mlsanityData to 14 Oct 2026

Partial Disentanglement via Mechanism Sparsity

arXiv 2207.07732 ↗First posted 15 Jul 2022stat.ML cs.LGcited by 29

Not on the map. Why?

Abstract (arXiv)

Disentanglement via mechanism sparsity was introduced recently as a principled approach to extract latent factors without supervision when the causal graph relating them in time is sparse, and/or when actions are observed and affect them sparsely. However, this theory applies only to ground-truth graphs satisfying a specific criterion. In this work, we introduce a generalization of this theory which applies to any ground-truth graph and specifies qualitatively how disentangled the learned representation is expected to be, via a new equivalence relation over models we call consistency. This equivalence captures which factors are expected to remain entangled and which are not based on the specific form of the ground-truth graph. We call this weaker form of identifiability partial disentanglement. The graphical criterion that allows complete disentanglement, proposed in an earlier work, can be derived as a special case of our theory. Finally, we enforce graph sparsity with constrained optimization and illustrate our theory and algorithm in simulations.

Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). Builds on 18 of its 35 references that are in the map. First posted on arXiv 15 Jul 2022.

Builds on

18 of 35 references.

References · 18

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2013Auto-Encoding Variational Bayescited by 4,053
  3. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  4. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,148
  5. 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 503
  6. 2021Self-Supervised Learning with Data Augmentations Provably Isolates Content from Stylecited by 312
  7. 2018Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learningcited by 282
  8. 2020Inductive Biases for Deep Learning of Higher-Level Cognitioncited by 275
  9. 2016Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICAcited by 273
  10. 2020Weakly-Supervised Disentanglement Without Compromisescited by 239
  11. 2021Towards Causal Representation Learningcited by 209
  12. 2021Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICAcited by 160
6 more
  1. 2022CITRIS: Causal Identifiability from Temporal Intervened Sequencescited by 133
  2. 2020ICE-BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICAcited by 102
  3. 2021Learning Temporally Causal Latent Processes from General Temporal Datacited by 98
  4. 2022Weakly Supervised Representation Learning with Sparse Perturbationscited by 65
  5. 2019Unsupervised Model Selection for Variational Disentangled Representation Learningcited by 46
  6. 2021Properties from Mechanisms: An Equivariance Perspective on Identifiable Representation Learningcited by 30

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 9; 2024: 6; 2025: 8; 2026: 4.

29 papers.

Sort:

By area:

  1. 2023Nonparametric Identifiability of Causal Representations from Unknown Interventionscited by 106
  2. 2023Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolationcited by 58
  3. 2023Causal Component Analysiscited by 56
  4. 2023Provably Learning Object-Centric Representationscited by 45
  5. 2022Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learningcited by 44
  6. 2023BISCUIT: Causal Representation Learning from Binary Interactionscited by 38
  7. 2022Learning Causal Representations of Single Cells via Sparse Mechanism Shift Modelingcited by 34
  8. 2023Generalizing Nonlinear ICA Beyond Structural Sparsitycited by 34
  9. 2023Multi-Domain Causal Representation Learning via Weak Distributional Invariancescited by 23
  10. 2024Marrying Causal Representation Learning with Dynamical Systems for Sciencecited by 23
  11. 2023Object-centric architectures enable efficient causal representation learningcited by 21
  12. 2023Identifying Linearly-Mixed Causal Representations from Multi-Node Interventionscited by 18
17 more
  1. 2024On the Identification of Temporally Causal Representation with Instantaneous Dependencecited by 16
  2. 2024Identifiable Causal Representation Learning: Unsupervised, Multi-View, and Multi-Environmentcited by 12
  3. 2025Cooper: A Library for Constrained Optimization in Deep Learningcited by 11
  4. 2024Language Agents Meet Causality -- Bridging LLMs and Causal World Modelscited by 9
  5. 2025Debiasing Reward Models by Representation Learning with Guaranteescited by 6
  6. 2025Sanity Checking Causal Representation Learning on a Simple Real-World Systemcited by 5
  7. 2024Toward the Identifiability of Comparative Deep Generative Modelscited by 4
  8. 2024From Orthogonality to Dependency: Learning Disentangled Representation for Multi-Modal Time-Series Sensing Signalscited by 4
  9. 2025CausalVerse: Benchmarking Causal Representation Learning with Configurable High-Fidelity Simulationscited by 4
  10. 2025Mechanistic Independence: A Principle for Identifiable Disentangled Representationscited by 3
  11. 2026Disentangling Dynamical Systems: Causal Representation Learning Meets Local Sparse Attentioncited by 3
  12. 2025Supervised Contrastive Block Disentanglementcited by 2
  13. 2025Online Time Series Forecasting with Theoretical Guaranteescited by 2
  14. 2025Causal View of Time Series Imputation: Some Identification Results on Missing Mechanismcited by 1
  15. 2026Hierarchical Action Learning for Weakly-Supervised Action Segmentationcited by 1
  16. 2026Multimodal LLMs under Pairwise Modalitiescited by 0
  17. 2026Hessian Rank Constraint for Learning Structure of Nonlinear Latent Variable Modelscited by 0