# Independent mechanism analysis, a new concept?

arXiv: https://arxiv.org/abs/2106.05200 · HTML: https://mlsanity.com/paper/2106.05200/

## Key facts

- Cited by 97 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 131 references that are in the map.
- First posted on arXiv 9 Jun 2021.

## Builds on (most cited first, 30 of 38)

- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Causal inference using invariant prediction: identification and confidence intervals](https://mlsanity.com/paper/1501.01332/) (2015)
- [Variational Autoencoders and Nonlinear ICA: A Unifying Framework](https://mlsanity.com/paper/1907.04809/) (2019)
- [Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style](https://mlsanity.com/paper/2106.04619/) (2021)
- [Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning](https://mlsanity.com/paper/1805.08651/) (2018)
- [Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA](https://mlsanity.com/paper/1605.06336/) (2016)
- [Residual Flows for Invertible Generative Modeling](https://mlsanity.com/paper/1906.02735/) (2019)
- [Causal Discovery with Continuous Additive Noise Models](https://mlsanity.com/paper/1309.6779/) (2013)
- [Recurrent Independent Mechanisms](https://mlsanity.com/paper/1909.10893/) (2019)
- [Neural Autoregressive Flows](https://mlsanity.com/paper/1804.00779/) (2018)
- [Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributions](https://mlsanity.com/paper/1707.06422/) (2017)
- [Anchor regression: heterogeneous data meets causality](https://mlsanity.com/paper/1801.06229/) (2018)
- [Learning Independent Causal Mechanisms](https://mlsanity.com/paper/1712.00961/) (2017)
- [Causal Autoregressive Flows](https://mlsanity.com/paper/2011.02268/) (2020)
- [The Hessian Penalty: A Weak Prior for Unsupervised Disentanglement](https://mlsanity.com/paper/2008.10599/) (2020)
- [The Incomplete Rosetta Stone Problem: Identifiability Results for Multi-View Nonlinear ICA](https://mlsanity.com/paper/1905.06642/) (2019)
- [Woulda, Coulda, Shoulda: Counterfactually-Guided Policy Search](https://mlsanity.com/paper/1811.06272/) (2018)
- [Hidden Markov Nonlinear ICA: Unsupervised Learning from Nonstationary Time Series](https://mlsanity.com/paper/2006.12107/) (2020)
- [External Validity: From Do-Calculus to Transportability Across Populations](https://mlsanity.com/paper/1503.01603/) (2015)
- [Sample-Efficient Reinforcement Learning via Counterfactual-Based Data Augmentation](https://mlsanity.com/paper/2012.09092/) (2020)
- [Causal Discovery with General Non-Linear Relationships Using Non-Linear ICA](https://mlsanity.com/paper/1904.09096/) (2019)
- [Blessing of dimensionality: mathematical foundations of the statistical physics of data](https://mlsanity.com/paper/1801.03421/) (2018)
- [Counterfactuals uncover the modular structure of deep generative models](https://mlsanity.com/paper/1812.03253/) (2018)
- [Towards causal generative scene models via competition of experts](https://mlsanity.com/paper/2004.12906/) (2020)
- [Deconfounding Reinforcement Learning in Observational Settings](https://mlsanity.com/paper/1812.10576/) (2018)
- [Robust Learning with the Hilbert-Schmidt Independence Criterion](https://mlsanity.com/paper/1910.00270/) (2019)
- [Distinguishing cause from effect using observational data: methods and benchmarks](https://mlsanity.com/paper/1412.3773/) (2014)
- [A Spectral Regularizer for Unsupervised Disentanglement](https://mlsanity.com/paper/1812.01161/) (2018)

## Built on by (most cited first, 30 of 97)

- [Weakly supervised causal representation learning](https://mlsanity.com/paper/2203.16437/) (2022)
- [CITRIS: Causal Identifiability from Temporal Intervened Sequences](https://mlsanity.com/paper/2202.03169/) (2022)
- [Nonparametric Identifiability of Causal Representations from Unknown Interventions](https://mlsanity.com/paper/2306.00542/) (2023)
- [Learning Linear Causal Representations from Interventions under General Nonlinear Mixing](https://mlsanity.com/paper/2306.02235/) (2023)
- [On the Identifiability of Nonlinear ICA: Sparsity and Beyond](https://mlsanity.com/paper/2206.07751/) (2022)
- [Identifiability of deep generative models without auxiliary information](https://mlsanity.com/paper/2206.10044/) (2022)
- [Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcuts](https://mlsanity.com/paper/2305.19951/) (2023)
- [Identifiability of latent-variable and structural-equation models: from linear to nonlinear](https://mlsanity.com/paper/2302.02672/) (2023)
- [Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolation](https://mlsanity.com/paper/2307.02598/) (2023)
- [Causal Component Analysis](https://mlsanity.com/paper/2305.17225/) (2023)
- [Function Classes for Identifiable Nonlinear Independent Component Analysis](https://mlsanity.com/paper/2208.06406/) (2022)
- [Causal Representation Learning for Instantaneous and Temporal Effects in Interactive Systems](https://mlsanity.com/paper/2206.06169/) (2022)
- [Mechanistic Mode Connectivity](https://mlsanity.com/paper/2211.08422/) (2022)
- [Nonlinear Independent Component Analysis for Principled Disentanglement in Unsupervised Deep Learning](https://mlsanity.com/paper/2303.16535/) (2023)
- [Provably Learning Object-Centric Representations](https://mlsanity.com/paper/2305.14229/) (2023)
- [Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learning](https://mlsanity.com/paper/2211.14666/) (2022)
- [Causal Discovery from Temporal Data: An Overview and New Perspectives](https://mlsanity.com/paper/2303.10112/) (2023)
- [Provable Compositional Generalization for Object-Centric Learning](https://mlsanity.com/paper/2310.05327/) (2023)
- [From Statistical to Causal Learning](https://mlsanity.com/paper/2204.00607/) (2022)
- [Disentanglement via Latent Quantization](https://mlsanity.com/paper/2305.18378/) (2023)
- [BISCUIT: Causal Representation Learning from Binary Interactions](https://mlsanity.com/paper/2306.09643/) (2023)
- [BEARS Make Neuro-Symbolic Models Aware of their Reasoning Shortcuts](https://mlsanity.com/paper/2402.12240/) (2024)
- [Emergence of Hidden Capabilities: Exploring Learning Dynamics in Concept Space](https://mlsanity.com/paper/2406.19370/) (2024)
- [Indeterminacy in Generative Models: Characterization and Strong Identifiability](https://mlsanity.com/paper/2206.00801/) (2022)
- [Causal Representation Learning Made Identifiable by Grouping of Observational Variables](https://mlsanity.com/paper/2310.15709/) (2023)
- [Properties from Mechanisms: An Equivariance Perspective on Identifiable Representation Learning](https://mlsanity.com/paper/2110.15796/) (2021)
- [Learning Interpretable Concepts: Unifying Causal Representation Learning and Foundation Models](https://mlsanity.com/paper/2402.09236/) (2024)
- [Embrace the Gap: VAEs Perform Independent Mechanism Analysis](https://mlsanity.com/paper/2206.02416/) (2022)
- [From Identifiable Causal Representations to Controllable Counterfactual Generation: A Survey on Causal Generative Modeling](https://mlsanity.com/paper/2310.11011/) (2023)
- [Object-centric architectures enable efficient causal representation learning](https://mlsanity.com/paper/2310.19054/) (2023)

## Abstract (arXiv)

Independent component analysis provides a principled framework for unsupervised representation learning, with solid theory on the identifiability of the latent code that generated the data, given only observations of mixtures thereof. Unfortunately, when the mixing is nonlinear, the model is provably nonidentifiable, since statistical independence alone does not sufficiently constrain the problem. Identifiability can be recovered in settings where additional, typically observed variables are included in the generative process. We investigate an alternative path and consider instead including assumptions reflecting the principle of independent causal mechanisms exploited in the field of causality. Specifically, our approach is motivated by thinking of each source as independently influencing the mixing process. This gives rise to a framework which we term independent mechanism analysis. We provide theoretical and empirical evidence that our approach circumvents a number of nonidentifiability issues arising in nonlinear blind source separation.