# Out-of-distribution Generalization with Causal Invariant Transformations

arXiv: https://arxiv.org/abs/2203.11528 · HTML: https://mlsanity.com/paper/2203.11528/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 82 references that are in the map.
- First posted on arXiv 22 Mar 2022.

## Builds on (most cited first, 30 of 51)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [TinyBERT: Distilling BERT for Natural Language Understanding](https://mlsanity.com/paper/1909.10351/) (2019)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)
- [Domain Generalization: A Survey](https://mlsanity.com/paper/2103.02503/) (2021)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks](https://mlsanity.com/paper/1703.10593/) (2017)
- [Domain Generalization via Invariant Feature Representation](https://mlsanity.com/paper/1301.2115/) (2013)
- [Causal inference using invariant prediction: identification and confidence intervals](https://mlsanity.com/paper/1501.01332/) (2015)
- [Generalizing to Unseen Domains via Adversarial Data Augmentation](https://mlsanity.com/paper/1805.12018/) (2018)
- [Learning the Difference that Makes a Difference with Counterfactually-Augmented Data](https://mlsanity.com/paper/1909.12434/) (2019)
- [Certifying Some Distributional Robustness with Principled Adversarial Training](https://mlsanity.com/paper/1710.10571/) (2017)
- [Learning Models with Uniform Performance via Distributionally Robust Optimization](https://mlsanity.com/paper/1810.08750/) (2018)
- [Learning to Generate Novel Domains for Domain Generalization](https://mlsanity.com/paper/2007.03304/) (2020)
- [FreeLB: Enhanced Adversarial Training for Natural Language Understanding](https://mlsanity.com/paper/1909.11764/) (2019)
- [The Risks of Invariant Risk Minimization](https://mlsanity.com/paper/2010.05761/) (2020)
- [Causality for Machine Learning](https://mlsanity.com/paper/1911.10500/) (2019)
- [Deep Domain-Adversarial Image Generation for Domain Generalisation](https://mlsanity.com/paper/2003.06054/) (2020)
- [Domain Generalization using Causal Matching](https://mlsanity.com/paper/2006.07500/) (2020)
- [Representation Learning via Invariant Causal Mechanisms](https://mlsanity.com/paper/2010.07922/) (2020)
- [Invariant Risk Minimization Games](https://mlsanity.com/paper/2002.04692/) (2020)

## Built on by (most cited first, 30 of 45)

- [A Sentence Speaks a Thousand Images: Domain Generalization through Distilling CLIP with Language Guidance](https://mlsanity.com/paper/2309.12530/) (2023)
- [Out-of-Domain Robustness via Targeted Augmentations](https://mlsanity.com/paper/2302.11861/) (2023)
- [Mitigating Data Heterogeneity in Federated Learning with Data Augmentation](https://mlsanity.com/paper/2206.09979/) (2022)
- [Causal Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2310.07379/) (2023)
- [Not Just Pretty Pictures: Toward Interventional Data Augmentation Using Text-to-Image Generators](https://mlsanity.com/paper/2212.11237/) (2022)
- [Causally Reliable Concept Bottleneck Models](https://mlsanity.com/paper/2503.04363/) (2025)
- [Meta-Causal Feature Learning for Out-of-Distribution Generalization](https://mlsanity.com/paper/2208.10156/) (2022)
- [Explore and Exploit the Diverse Knowledge in Model Zoo for Domain Generalization](https://mlsanity.com/paper/2306.02595/) (2023)
- [Latent Covariate Shift: Unlocking Partial Identifiability for Multi-Source Domain Adaptation](https://mlsanity.com/paper/2208.14161/) (2022)
- [Enhancing Adversarial Contrastive Learning via Adversarial Invariant Regularization](https://mlsanity.com/paper/2305.00374/) (2023)
- [Causal Inference via Style Transfer for Out-of-distribution Generalisation](https://mlsanity.com/paper/2212.03063/) (2022)
- [Invariant Anomaly Detection under Distribution Shifts: A Causal Perspective](https://mlsanity.com/paper/2312.14329/) (2023)
- [Breaking Correlation Shift via Conditional Invariant Regularizer](https://mlsanity.com/paper/2207.06687/) (2022)
- [Non-parametric identifiability and sensitivity analysis of synthetic control models](https://mlsanity.com/paper/2301.07656/) (2023)
- [Unified Source-Free Domain Adaptation](https://mlsanity.com/paper/2403.07601/) (2024)
- [Spatio-Temporal Fluid Dynamics Modeling via Physical-Awareness and Parameter Diffusion Guidance](https://mlsanity.com/paper/2403.13850/) (2024)
- [Invariant Shape Representation Learning For Image Classification](https://mlsanity.com/paper/2411.12201/) (2024)
- [Scalable Out-of-distribution Robustness in the Presence of Unobserved Confounders](https://mlsanity.com/paper/2411.19923/) (2024)
- [Learning Time-Aware Causal Representation for Model Generalization in Evolving Domains](https://mlsanity.com/paper/2506.17718/) (2025)
- [Consistency Regularization for Domain Generalization with Logit Attribution Matching](https://mlsanity.com/paper/2305.07888/) (2023)
- [A Causal Inspired Early-Branching Structure for Domain Generalization](https://mlsanity.com/paper/2403.08649/) (2024)
- [CausalDiff: Causality-Inspired Disentanglement via Diffusion Model for Adversarial Defense](https://mlsanity.com/paper/2410.23091/) (2024)
- [A Causal Framework to Unify Common Domain Generalization Approaches](https://mlsanity.com/paper/2307.06825/) (2023)
- [Improved Diffusion-based Generative Model with Better Adversarial Robustness](https://mlsanity.com/paper/2502.17099/) (2025)
- [Harnessing Synthetic Data from Generative AI for Statistical Inference](https://mlsanity.com/paper/2603.05396/) (2026)
- [Counterfactual Generation Under Confounding](https://mlsanity.com/paper/2210.12368/) (2022)
- [Causal Invariance Learning via Efficient Nonconvex Optimization](https://mlsanity.com/paper/2412.11850/) (2024)
- [Why Domain Generalization Fail? A View of Necessity and Sufficiency](https://mlsanity.com/paper/2502.10716/) (2025)
- [Deconfounding Causal Inference through Two-Branch Framework with Early-Forking for Sensor-Based Cross-Domain Activity Recognition](https://mlsanity.com/paper/2507.03898/) (2025)
- [An Analysis of Causal Effect Estimation using Outcome Invariant Data Augmentation](https://mlsanity.com/paper/2510.25128/) (2025)

## Abstract (arXiv)

In real-world applications, it is important and desirable to learn a model that performs well on out-of-distribution (OOD) data. Recently, causality has become a powerful tool to tackle the OOD generalization problem, with the idea resting on the causal mechanism that is invariant across domains of interest. To leverage the generally unknown causal mechanism, existing works assume a linear form of causal feature or require sufficiently many and diverse training domains, which are usually restrictive in practice. In this work, we obviate these assumptions and tackle the OOD problem without explicitly recovering the causal feature. Our approach is based on transformations that modify the non-causal feature but leave the causal part unchanged, which can be either obtained from prior knowledge or learned from the training data in the multi-domain scenario. Under the setting of invariant causal mechanism, we theoretically show that if all such transformations are available, then we can learn a minimax optimal model across the domains using only single domain data. Noticing that knowing a complete set of these causal invariant transformations may be impractical, we further show that it suffices to know only a subset of these transformations. Based on the theoretical findings, a regularized training procedure is proposed to improve the OOD generalization capability. Extensive experimental results on both synthetic and real datasets verify the effectiveness of the proposed algorithm, even with only a few causal invariant transformations.