# Learning Robust Models Using The Principle of Independent Causal Mechanisms

arXiv: https://arxiv.org/abs/2010.07167 · HTML: https://mlsanity.com/paper/2010.07167/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Oct 2020.

## Built on by (most cited first, 18 of 18)

- [Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization](https://mlsanity.com/paper/2106.06607/) (2021)
- [Towards a Theoretical Framework of Out-of-Distribution Generalization](https://mlsanity.com/paper/2106.04496/) (2021)
- [Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?](https://mlsanity.com/paper/2106.02890/) (2021)
- [WOODS: Benchmarks for Out-of-Distribution Generalization in Time Series](https://mlsanity.com/paper/2203.09978/) (2022)
- [Normalizing Flows for Interventional Density Estimation](https://mlsanity.com/paper/2209.06203/) (2022)
- [Causal Deep Learning](https://mlsanity.com/paper/2303.02186/) (2023)
- [On the benefits of representation regularization in invariance based domain generalization](https://mlsanity.com/paper/2105.14529/) (2021)
- [Learning Representations that Support Robust Transfer of Predictors](https://mlsanity.com/paper/2110.09940/) (2021)
- [Optimization-based Causal Estimation from Heterogenous Environments](https://mlsanity.com/paper/2109.11990/) (2021)
- [Domain Generalization -- A Causal Perspective](https://mlsanity.com/paper/2209.15177/) (2022)
- [Towards Artificial General Intelligence (AGI) in the Internet of Things (IoT): Opportunities and Challenges](https://mlsanity.com/paper/2309.07438/) (2023)
- [Whitening Convergence Rate of Coupling-based Normalizing Flows](https://mlsanity.com/paper/2210.14032/) (2022)
- [Finding Competence Regions in Domain Generalization](https://mlsanity.com/paper/2303.09989/) (2023)
- [Bridging Distribution Shift and AI Safety: Conceptual and Methodological Synergies](https://mlsanity.com/paper/2505.22829/) (2025)
- [Navigating causal deep learning](https://mlsanity.com/paper/2212.00911/) (2022)
- [Context is Environment](https://mlsanity.com/paper/2309.09888/) (2023)
- [Catastrophic Compositional Generation: Why Vanilla Diffusion Models Fail to Extrapolate](https://mlsanity.com/paper/2606.23920/) (2026)
- [Towards Context-Aware Domain Generalization: Understanding the Benefits and Limits of Marginal Transfer Learning](https://mlsanity.com/paper/2312.10107/) (2023)

## Abstract (arXiv)

Standard supervised learning breaks down under data distribution shift. However, the principle of independent causal mechanisms (ICM, Peters et al. (2017)) can turn this weakness into an opportunity: one can take advantage of distribution shift between different environments during training in order to obtain more robust models. We propose a new gradient-based learning framework whose objective function is derived from the ICM principle. We show theoretically and experimentally that neural networks trained in this framework focus on relations remaining invariant across environments and ignore unstable ones. Moreover, we prove that the recovered stable relations correspond to the true causal mechanisms under certain conditions. In both regression and classification, the resulting models generalize well to unseen scenarios where traditionally trained models fail.