# Invariant Causal Mechanisms through Distribution Matching

arXiv: https://arxiv.org/abs/2206.11646 · HTML: https://mlsanity.com/paper/2206.11646/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 55 references that are in the map.
- First posted on arXiv 23 Jun 2022.

## Builds on (most cited first, 30 of 34)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Recognition in Terra Incognita](https://mlsanity.com/paper/1807.04975/) (2018)
- [Domain Generalization via Invariant Feature Representation](https://mlsanity.com/paper/1301.2115/) (2013)
- [Causal inference using invariant prediction: identification and confidence intervals](https://mlsanity.com/paper/1501.01332/) (2015)
- [Self-Challenging Improves Cross-Domain Generalization](https://mlsanity.com/paper/2007.02454/) (2020)
- [Domain Generalization for Object Recognition with Multi-task Autoencoders](https://mlsanity.com/paper/1508.07680/) (2015)
- [DeepJDOT: Deep Joint Distribution Optimal Transport for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1803.10081/) (2018)
- [Towards Causal Representation Learning](https://mlsanity.com/paper/2102.11107/) (2021)
- [Flexibly Fair Representation Learning by Disentanglement](https://mlsanity.com/paper/1906.02589/) (2019)
- [Censoring Representations with an Adversary](https://mlsanity.com/paper/1511.05897/) (2015)
- [Improve Unsupervised Domain Adaptation with Mixup Training](https://mlsanity.com/paper/2001.00677/) (2020)
- [On the Fairness of Disentangled Representations](https://mlsanity.com/paper/1905.13662/) (2019)
- [Wasserstein Fair Classification](https://mlsanity.com/paper/1907.12059/) (2019)
- [Controllable Invariance through Adversarial Feature Learning](https://mlsanity.com/paper/1705.11122/) (2017)
- [Anchor regression: heterogeneous data meets causality](https://mlsanity.com/paper/1801.06229/) (2018)
- [Generalizing to unseen domains via distribution matching](https://mlsanity.com/paper/1911.00804/) (2019)
- [Robustly Disentangled Causal Mechanisms: Validating Deep Representations for Interventional Robustness](https://mlsanity.com/paper/1811.00007/) (2018)
- [On Disentangled Representations Learned From Correlated Data](https://mlsanity.com/paper/2006.07886/) (2020)
- [Mitigating Information Leakage in Image Representations: A Maximum Entropy Approach](https://mlsanity.com/paper/1904.05514/) (2019)

## Built on by (most cited first, 30 of 31)

- [Spawrious: A Benchmark for Fine Control of Spurious Correlation Biases](https://mlsanity.com/paper/2303.05470/) (2023)
- [Domain Generalization in Computational Pathology: Survey and Guidelines](https://mlsanity.com/paper/2310.19656/) (2023)
- [Domain Generalization -- A Causal Perspective](https://mlsanity.com/paper/2209.15177/) (2022)
- [Domain Generalisation via Risk Distribution Matching](https://mlsanity.com/paper/2310.18598/) (2023)
- [Back-to-Bones: Rediscovering the Role of Backbones in Domain Generalization](https://mlsanity.com/paper/2209.01121/) (2022)
- [HCVP: Leveraging Hierarchical Contrastive Visual Prompt for Domain Generalization](https://mlsanity.com/paper/2401.09716/) (2024)
- [Controlled Training Data Generation with Diffusion Models](https://mlsanity.com/paper/2403.15309/) (2024)
- [Non-parametric identifiability and sensitivity analysis of synthetic control models](https://mlsanity.com/paper/2301.07656/) (2023)
- [CauDR: A Causality-inspired Domain Generalization Framework for Fundus-based Diabetic Retinopathy Grading](https://mlsanity.com/paper/2309.15493/) (2023)
- [Balanced Direction from Multifarious Choices: Arithmetic Meta-Learning for Domain Generalization](https://mlsanity.com/paper/2503.18987/) (2025)
- [When Shift Happens - Confounding Is to Blame](https://mlsanity.com/paper/2505.21422/) (2025)
- [Quantitatively Measuring and Contrastively Exploring Heterogeneity for Domain Generalization](https://mlsanity.com/paper/2305.15889/) (2023)
- [Causally Inspired Regularization Enables Domain General Representations](https://mlsanity.com/paper/2404.16277/) (2024)
- [Conditionally Invariant Representation Learning for Disentangling Cellular Heterogeneity](https://mlsanity.com/paper/2307.00558/) (2023)
- [How Does Distribution Matching Help Domain Generalization: An Information-theoretic Analysis](https://mlsanity.com/paper/2406.09745/) (2024)
- [Benchmarking Domain Generalization Algorithms in Computational Pathology](https://mlsanity.com/paper/2409.17063/) (2024)
- [Robustness May be More Brittle than We Think under Different Degrees of Distribution Shifts](https://mlsanity.com/paper/2310.06622/) (2023)
- [UnLearning from Experience to Avoid Spurious Correlations](https://mlsanity.com/paper/2409.02792/) (2024)
- [Bridging Distribution Shift and AI Safety: Conceptual and Methodological Synergies](https://mlsanity.com/paper/2505.22829/) (2025)
- [Learning Invariant Representations with a Nonparametric Nadaraya-Watson Head](https://mlsanity.com/paper/2309.13377/) (2023)
- [Diagnosing and Rectifying Fake OOD Invariance: A Restructured Causal Approach](https://mlsanity.com/paper/2312.09758/) (2023)
- [Revisiting Spurious Correlation in Domain Generalization](https://mlsanity.com/paper/2406.11517/) (2024)
- [C3DG: Conditional Domain Generalization for Hyperspectral Imagery Classification with Convergence and Constrained-risk Theories](https://mlsanity.com/paper/2407.04100/) (2024)
- [DICS: Find Domain-Invariant and Class-Specific Features for Out-of-Distribution Generalization](https://mlsanity.com/paper/2409.08557/) (2024)
- [Robust Context-Aware Object Recognition](https://mlsanity.com/paper/2510.00618/) (2025)
- [Exploring Scale Shift in Crowd Localization under the Context of Domain Generalization](https://mlsanity.com/paper/2510.19330/) (2025)
- [A2: Smaller Self-Supervised ViTs Localize Better than Larger Ones](https://mlsanity.com/paper/2606.03148/) (2026)
- [Direct-Effect Risk Minimization for Domain Generalization](https://mlsanity.com/paper/2211.14594/) (2022)
- [Bringing the Context Back into Object Recognition, Robustly](https://mlsanity.com/paper/2411.15933/) (2024)
- [Feature Matching Intervention: Leveraging Observational Data for Causal Representation Learning](https://mlsanity.com/paper/2503.03634/) (2025)

## Abstract (arXiv)

Learning representations that capture the underlying data generating process is a key problem for data efficient and robust use of neural networks. One key property for robustness which the learned representation should capture and which recently received a lot of attention is described by the notion of invariance. In this work we provide a causal perspective and new algorithm for learning invariant representations. Empirically we show that this algorithm works well on a diverse set of tasks and in particular we observe state-of-the-art performance on domain generalization, where we are able to significantly boost the score of existing models.