mlsanityData to 13 Oct 2026

Properties from Mechanisms: An Equivariance Perspective on Identifiable Representation Learning

arXiv 2110.15796 ↗First posted 29 Oct 2021cs.LG cs.AI stat.MLcited by 30

Not on the map. Why?

Abstract (arXiv)

A key goal of unsupervised representation learning is "inverting" a data generating process to recover its latent properties. Existing work that provably achieves this goal relies on strong assumptions on relationships between the latent variables (e.g., independence conditional on auxiliary information). In this paper, we take a very different perspective on the problem and ask, "Can we instead identify latent properties by leveraging knowledge of the mechanisms that govern their evolution?" We provide a complete characterization of the sources of non-identifiability as we vary knowledge about a set of possible mechanisms. In particular, we prove that if we know the exact mechanisms under which the latent properties evolve, then identification can be achieved up to any equivariances that are shared by the underlying mechanisms. We generalize this characterization to settings where we only know some hypothesis class over possible mechanisms, as well as settings where the mechanisms are stochastic. We demonstrate the power of this mechanism-based perspective by showing that we can leverage our results to generalize existing identifiable representation learning results. These results suggest that by exploiting inductive biases on mechanisms, it is possible to design a range of new identifiable representation learning approaches.

Key facts. Cited by 30 papers in the map (data through 13 Oct 2026). Builds on 18 of its 34 references that are in the map. First posted on arXiv 29 Oct 2021.

Builds on

18 of 34 references.

References · 18

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,028
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,000
  4. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,381
  5. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,147
  6. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 934
  7. 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 498
  8. 2018Towards a Definition of Disentangled Representationscited by 295
  9. 2018Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learningcited by 279
  10. 2016Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICAcited by 272
  11. 2020Weakly-Supervised Disentanglement Without Compromisescited by 237
  12. 2021Contrastive Learning Inverts the Data Generating Processcited by 220
6 more
  1. 2021Towards Causal Representation Learningcited by 209
  2. 2020Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Codingcited by 122
  3. 2020ICE-BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICAcited by 102
  4. 2021Independent mechanism analysis, a new concept?cited by 97
  5. 2019Weakly Supervised Disentanglement with Guaranteescited by 82
  6. 2020Hidden Markov Nonlinear ICA: Unsupervised Learning from Nonstationary Time Seriescited by 76

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2022: 7; 2023: 9; 2024: 7; 2025: 3; 2026: 4.

30 papers.

Sort:

By area:

  1. 2022Interventional Causal Representation Learningcited by 141
  2. 2023Nonparametric Identifiability of Causal Representations from Unknown Interventionscited by 105
  3. 2022Linear Causal Disentanglement via Interventionscited by 84
  4. 2023Identifiability Guarantees for Causal Disentanglement from Soft Interventionscited by 84
  5. 2022Weakly Supervised Representation Learning with Sparse Perturbationscited by 64
  6. 2023Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolationcited by 56
  7. 2023Score-based Causal Representation Learning with Interventionscited by 49
  8. 2022Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learningcited by 44
  9. 2023General Identifiability and Achievability for Causal Representation Learningcited by 34
  10. 2022Indeterminacy in Generative Models: Characterization and Strong Identifiabilitycited by 33
  11. 2022Partial Disentanglement via Mechanism Sparsitycited by 29
  12. 2025A Closer Look at Multimodal Representation Collapsecited by 28
18 more
  1. 2023Multi-Domain Causal Representation Learning via Weak Distributional Invariancescited by 23
  2. 2023Object-centric architectures enable efficient causal representation learningcited by 21
  3. 2022Variational Causal Dynamics: Discovering Modular World Models from Interventionscited by 16
  4. 2025Towards Interpretable Deep Generative Models via Causal Representation Learningcited by 13
  5. 2024Identifiable Causal Representation Learning: Unsupervised, Multi-View, and Multi-Environmentcited by 12
  6. 2025What is causal about causal models and representations?cited by 6
  7. 2024Causal Representation Learning in Temporal Data via Single-Parent Decodingcited by 5
  8. 2026Disentangling Dynamical Systems: Causal Representation Learning Meets Local Sparse Attentioncited by 3
  9. 2024Sparsity regularization via tree-structured environments for disentangled representationscited by 2
  10. 2024Bayesian Unsupervised Disentanglement of Anatomy and Geometry for Deep Groupwise Image Registrationcited by 1
  11. 2024Linear causal disentanglement via higher-order cumulantscited by 1
  12. 2024Identifiable Representation and Model Learning for Latent Dynamic Systemscited by 1
  13. 2026Concept Modulation Models: A Unified Framework for Identifiability and Extrapolationcited by 1
  14. 2023Equivariant Representation Learning in the Presence of Stabilizerscited by 0
  15. 2023Interpretable ODE-style Generative Diffusion Model via Force Field Constructioncited by 0
  16. 2024Interpretable Representation Learning from Videos using Nonlinear Priorscited by 0
  17. 2026Discrete Causal Representations from Heterogeneous Domains: A Bayesian Approach with Social Survey Applicationscited by 0
  18. 2026Beyond ICA: Identifiability by Symmetry Breakingcited by 0