# Causal Discovery in Heterogeneous Environments Under the Sparse Mechanism Shift Hypothesis

arXiv: https://arxiv.org/abs/2206.02013 · HTML: https://mlsanity.com/paper/2206.02013/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 67 references that are in the map.
- First posted on arXiv 4 Jun 2022.

## Builds on (most cited first, 26 of 26)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Recognition in Terra Incognita](https://mlsanity.com/paper/1807.04975/) (2018)
- [Causal inference using invariant prediction: identification and confidence intervals](https://mlsanity.com/paper/1501.01332/) (2015)
- [Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style](https://mlsanity.com/paper/2106.04619/) (2021)
- [Environment Inference for Invariant Learning](https://mlsanity.com/paper/2010.07249/) (2020)
- [Weakly-Supervised Disentanglement Without Compromises](https://mlsanity.com/paper/2002.02886/) (2020)
- [Differentiable Causal Discovery from Interventional Data](https://mlsanity.com/paper/2007.01754/) (2020)
- [Causal Inference in the Presence of Latent Variables and Selection Bias](https://mlsanity.com/paper/1302.4983/) (2013)
- [Causal Discovery from Heterogeneous/Nonstationary Data with Independent Changes](https://mlsanity.com/paper/1903.01672/) (2019)
- [Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICA](https://mlsanity.com/paper/2107.10098/) (2021)
- [A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms](https://mlsanity.com/paper/1901.10912/) (2019)
- [Invariant Causal Prediction for Nonlinear Models](https://mlsanity.com/paper/1706.08576/) (2017)
- [CITRIS: Causal Identifiability from Temporal Intervened Sequences](https://mlsanity.com/paper/2202.03169/) (2022)
- [Does Invariant Risk Minimization Capture Invariance?](https://mlsanity.com/paper/2101.01134/) (2021)
- [Learning Neural Causal Models from Unknown Interventions](https://mlsanity.com/paper/1910.01075/) (2019)
- [Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Tests](https://mlsanity.com/paper/2106.00545/) (2021)
- [Characterizing and Learning Equivalence Classes of Causal DAGs under Interventions](https://mlsanity.com/paper/1802.06310/) (2018)
- [Permutation-Based Causal Structure Learning with Unknown Intervention Targets](https://mlsanity.com/paper/1910.09007/) (2019)
- [Learning Causal Graphs with Small Interventions](https://mlsanity.com/paper/1511.00041/) (2015)
- [Causal Discovery from Changes](https://mlsanity.com/paper/1301.2312/) (2013)
- [Causal de Finetti: On the Identification of Invariant Causal Structure in Exchangeable Data](https://mlsanity.com/paper/2203.15756/) (2022)
- [Conditional Distributional Treatment Effect with Kernel Conditional Mean Embeddings and U-Statistic Regression](https://mlsanity.com/paper/2102.08208/) (2021)
- [Learning Causal Structures Using Regression Invariance](https://mlsanity.com/paper/1705.09644/) (2017)
- [FASK with Interventional Knowledge Recovers Edges from the Sachs Model](https://mlsanity.com/paper/1805.03108/) (2018)
- [Causal Network Learning from Multiple Interventions of Unknown Manipulated Targets](https://mlsanity.com/paper/1610.08611/) (2016)

## Built on by (most cited first, 30 of 55)

- [Nonparametric Identifiability of Causal Representations from Unknown Interventions](https://mlsanity.com/paper/2306.00542/) (2023)
- [Causal Component Analysis](https://mlsanity.com/paper/2305.17225/) (2023)
- [Score-based Causal Representation Learning with Interventions](https://mlsanity.com/paper/2301.08230/) (2023)
- [A Sparsity Principle for Partially Observable Causal Representation Learning](https://mlsanity.com/paper/2403.08335/) (2024)
- [Learning nonparametric latent causal graphs with unknown interventions](https://mlsanity.com/paper/2306.02899/) (2023)
- [Drift-Resilient TabPFN: In-Context Learning Temporal Distribution Shifts on Tabular Data](https://mlsanity.com/paper/2411.10634/) (2024)
- [Causal Reinforcement Learning: A Survey](https://mlsanity.com/paper/2307.01452/) (2023)
- [DGMamba: Domain Generalization via Generalized State Space Model](https://mlsanity.com/paper/2404.07794/) (2024)
- [Identifying Linearly-Mixed Causal Representations from Multi-Node Interventions](https://mlsanity.com/paper/2311.02695/) (2023)
- [Rethinking Domain Generalization: Discriminability and Generalizability](https://mlsanity.com/paper/2309.16483/) (2023)
- [Identifiable Exchangeable Mechanisms for Causal Structure and Representation Learning](https://mlsanity.com/paper/2406.14302/) (2024)
- [Towards Interpretable Deep Generative Models via Causal Representation Learning](https://mlsanity.com/paper/2504.11609/) (2025)
- [Identifiable Causal Representation Learning: Unsupervised, Multi-View, and Multi-Environment](https://mlsanity.com/paper/2406.13371/) (2024)
- [Deep Backtracking Counterfactuals for Causally Compliant Explanations](https://mlsanity.com/paper/2310.07665/) (2023)
- [An Interventional Perspective on Identifiability in Gaussian LTI Systems with Independent Component Analysis](https://mlsanity.com/paper/2311.18048/) (2023)
- [Learning Unknown Intervention Targets in Structural Causal Models from Heterogeneous Data](https://mlsanity.com/paper/2312.06091/) (2023)
- [MetaPhysiCa: OOD Robustness in Physics-informed Machine Learning](https://mlsanity.com/paper/2303.03181/) (2023)
- [Causal Graphical Models for Vision-Language Compositional Understanding](https://mlsanity.com/paper/2412.09353/) (2024)
- [SpaceTime: Causal Discovery from Non-Stationary Time Series](https://mlsanity.com/paper/2501.10235/) (2025)
- [Causally Aligned Curriculum Learning](https://mlsanity.com/paper/2503.16799/) (2025)
- [On the Interventional Kullback-Leibler Divergence](https://mlsanity.com/paper/2302.05380/) (2023)
- [iSCAN: Identifying Causal Mechanism Shifts among Nonlinear Additive Noise Models](https://mlsanity.com/paper/2306.17361/) (2023)
- [Out-of-Variable Generalization for Discriminative Models](https://mlsanity.com/paper/2304.07896/) (2023)
- [Multi-Domain Causal Discovery in Bijective Causal Models](https://mlsanity.com/paper/2504.21261/) (2025)
- [Learning Multiscale Non-stationary Causal Structures](https://mlsanity.com/paper/2208.14989/) (2022)
- [Detecting and Measuring Confounding Using Causal Mechanism Shifts](https://mlsanity.com/paper/2409.17840/) (2024)
- [Algorithmic causal structure emerging through compression](https://mlsanity.com/paper/2502.04210/) (2025)
- [Falsification of Unconfoundedness by Testing Independence of Causal Mechanisms](https://mlsanity.com/paper/2502.06231/) (2025)
- [CausalEvolve: Towards Open-Ended Discovery with Causal Scratchpad](https://mlsanity.com/paper/2603.14575/) (2026)
- [Exploiting Causality Signals in Medical Images: A Pilot Study with Empirical Results](https://mlsanity.com/paper/2309.10399/) (2023)

## Abstract (arXiv)

Machine learning approaches commonly rely on the assumption of independent and identically distributed (i.i.d.) data. In reality, however, this assumption is almost always violated due to distribution shifts between environments. Although valuable learning signals can be provided by heterogeneous data from changing distributions, it is also known that learning under arbitrary (adversarial) changes is impossible. Causality provides a useful framework for modeling distribution shifts, since causal models encode both observational and interventional distributions. In this work, we explore the sparse mechanism shift hypothesis, which posits that distribution shifts occur due to a small number of changing causal conditionals. Motivated by this idea, we apply it to learning causal structure from heterogeneous environments, where i.i.d. data only allows for learning an equivalence class of graphs without restrictive assumptions. We propose the Mechanism Shift Score (MSS), a score-based approach amenable to various empirical estimators, which provably identifies the entire causal structure with high probability if the sparse mechanism shift hypothesis holds. Empirically, we verify behavior predicted by the theory and compare multiple estimators and score functions to identify the best approaches in practice. Compared to other methods, we show how MSS bridges a gap by both being nonparametric as well as explicitly leveraging sparse changes.