# Properties from Mechanisms: An Equivariance Perspective on Identifiable Representation Learning

arXiv: https://arxiv.org/abs/2110.15796 · HTML: https://mlsanity.com/paper/2110.15796/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 34 references that are in the map.
- First posted on arXiv 29 Oct 2021.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Variational Autoencoders and Nonlinear ICA: A Unifying Framework](https://mlsanity.com/paper/1907.04809/) (2019)
- [Towards a Definition of Disentangled Representations](https://mlsanity.com/paper/1812.02230/) (2018)
- [Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning](https://mlsanity.com/paper/1805.08651/) (2018)
- [Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA](https://mlsanity.com/paper/1605.06336/) (2016)
- [Weakly-Supervised Disentanglement Without Compromises](https://mlsanity.com/paper/2002.02886/) (2020)
- [Contrastive Learning Inverts the Data Generating Process](https://mlsanity.com/paper/2102.08850/) (2021)
- [Towards Causal Representation Learning](https://mlsanity.com/paper/2102.11107/) (2021)
- [Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding](https://mlsanity.com/paper/2007.10930/) (2020)
- [ICE-BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICA](https://mlsanity.com/paper/2002.11537/) (2020)
- [Independent mechanism analysis, a new concept?](https://mlsanity.com/paper/2106.05200/) (2021)
- [Weakly Supervised Disentanglement with Guarantees](https://mlsanity.com/paper/1910.09772/) (2019)
- [Hidden Markov Nonlinear ICA: Unsupervised Learning from Nonstationary Time Series](https://mlsanity.com/paper/2006.12107/) (2020)

## Built on by (most cited first, 30 of 30)

- [Interventional Causal Representation Learning](https://mlsanity.com/paper/2209.11924/) (2022)
- [Nonparametric Identifiability of Causal Representations from Unknown Interventions](https://mlsanity.com/paper/2306.00542/) (2023)
- [Linear Causal Disentanglement via Interventions](https://mlsanity.com/paper/2211.16467/) (2022)
- [Identifiability Guarantees for Causal Disentanglement from Soft Interventions](https://mlsanity.com/paper/2307.06250/) (2023)
- [Weakly Supervised Representation Learning with Sparse Perturbations](https://mlsanity.com/paper/2206.01101/) (2022)
- [Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolation](https://mlsanity.com/paper/2307.02598/) (2023)
- [Score-based Causal Representation Learning with Interventions](https://mlsanity.com/paper/2301.08230/) (2023)
- [Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learning](https://mlsanity.com/paper/2211.14666/) (2022)
- [General Identifiability and Achievability for Causal Representation Learning](https://mlsanity.com/paper/2310.15450/) (2023)
- [Indeterminacy in Generative Models: Characterization and Strong Identifiability](https://mlsanity.com/paper/2206.00801/) (2022)
- [Partial Disentanglement via Mechanism Sparsity](https://mlsanity.com/paper/2207.07732/) (2022)
- [A Closer Look at Multimodal Representation Collapse](https://mlsanity.com/paper/2505.22483/) (2025)
- [Multi-Domain Causal Representation Learning via Weak Distributional Invariances](https://mlsanity.com/paper/2310.02854/) (2023)
- [Object-centric architectures enable efficient causal representation learning](https://mlsanity.com/paper/2310.19054/) (2023)
- [Variational Causal Dynamics: Discovering Modular World Models from Interventions](https://mlsanity.com/paper/2206.11131/) (2022)
- [Towards Interpretable Deep Generative Models via Causal Representation Learning](https://mlsanity.com/paper/2504.11609/) (2025)
- [Identifiable Causal Representation Learning: Unsupervised, Multi-View, and Multi-Environment](https://mlsanity.com/paper/2406.13371/) (2024)
- [What is causal about causal models and representations?](https://mlsanity.com/paper/2501.19335/) (2025)
- [Causal Representation Learning in Temporal Data via Single-Parent Decoding](https://mlsanity.com/paper/2410.07013/) (2024)
- [Disentangling Dynamical Systems: Causal Representation Learning Meets Local Sparse Attention](https://mlsanity.com/paper/2603.14483/) (2026)
- [Sparsity regularization via tree-structured environments for disentangled representations](https://mlsanity.com/paper/2405.20482/) (2024)
- [Bayesian Unsupervised Disentanglement of Anatomy and Geometry for Deep Groupwise Image Registration](https://mlsanity.com/paper/2401.02141/) (2024)
- [Linear causal disentanglement via higher-order cumulants](https://mlsanity.com/paper/2407.04605/) (2024)
- [Identifiable Representation and Model Learning for Latent Dynamic Systems](https://mlsanity.com/paper/2410.17882/) (2024)
- [Concept Modulation Models: A Unified Framework for Identifiability and Extrapolation](https://mlsanity.com/paper/2606.18509/) (2026)
- [Equivariant Representation Learning in the Presence of Stabilizers](https://mlsanity.com/paper/2301.05231/) (2023)
- [Interpretable ODE-style Generative Diffusion Model via Force Field Construction](https://mlsanity.com/paper/2303.08063/) (2023)
- [Interpretable Representation Learning from Videos using Nonlinear Priors](https://mlsanity.com/paper/2410.18539/) (2024)
- [Discrete Causal Representations from Heterogeneous Domains: A Bayesian Approach with Social Survey Applications](https://mlsanity.com/paper/2606.06288/) (2026)
- [Beyond ICA: Identifiability by Symmetry Breaking](https://mlsanity.com/paper/2607.23182/) (2026)

## Abstract (arXiv)

A key goal of unsupervised representation learning is "inverting" a data generating process to recover its latent properties. Existing work that provably achieves this goal relies on strong assumptions on relationships between the latent variables (e.g., independence conditional on auxiliary information). In this paper, we take a very different perspective on the problem and ask, "Can we instead identify latent properties by leveraging knowledge of the mechanisms that govern their evolution?" We provide a complete characterization of the sources of non-identifiability as we vary knowledge about a set of possible mechanisms. In particular, we prove that if we know the exact mechanisms under which the latent properties evolve, then identification can be achieved up to any equivariances that are shared by the underlying mechanisms. We generalize this characterization to settings where we only know some hypothesis class over possible mechanisms, as well as settings where the mechanisms are stochastic. We demonstrate the power of this mechanism-based perspective by showing that we can leverage our results to generalize existing identifiable representation learning results. These results suggest that by exploiting inductive biases on mechanisms, it is possible to design a range of new identifiable representation learning approaches.