# When is invariance useful in an Out-of-Distribution Generalization problem ?

arXiv: https://arxiv.org/abs/2008.01883 · HTML: https://mlsanity.com/paper/2008.01883/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Aug 2020.

## Built on by (most cited first, 30 of 52)

- [Handling Distribution Shifts on Graphs: An Invariance Perspective](https://mlsanity.com/paper/2202.02466/) (2022)
- [DrugOOD: Out-of-Distribution (OOD) Dataset Curator and Benchmark for AI-aided Drug Discovery -- A Focus on Affinity Prediction Problems with Noise Annotations](https://mlsanity.com/paper/2201.09637/) (2022)
- [Unleashing the Power of Graph Data Augmentation on Covariate Distribution Shift](https://mlsanity.com/paper/2211.02843/) (2022)
- [Quantifying and Improving Transferability in Domain Generalization](https://mlsanity.com/paper/2106.03632/) (2021)
- [WOODS: Benchmarks for Out-of-Distribution Generalization in Time Series](https://mlsanity.com/paper/2203.09978/) (2022)
- [Joint Learning of Label and Environment Causal Independence for Graph Out-of-Distribution Generalization](https://mlsanity.com/paper/2306.01103/) (2023)
- [Domain Generalization in Computational Pathology: Survey and Guidelines](https://mlsanity.com/paper/2310.19656/) (2023)
- [Improving Multi-Task Generalization via Regularizing Spurious Correlation](https://mlsanity.com/paper/2205.09797/) (2022)
- [Mitigating and Evaluating Static Bias of Action Representations in the Background and the Foreground](https://mlsanity.com/paper/2211.12883/) (2022)
- [Time-Series Forecasting for Out-of-Distribution Generalization Using Invariant Learning](https://mlsanity.com/paper/2406.09130/) (2024)
- [Distribution Shift Inversion for Out-of-Distribution Prediction](https://mlsanity.com/paper/2306.08328/) (2023)
- [Gradient Masked Averaging for Federated Learning](https://mlsanity.com/paper/2201.11986/) (2022)
- [A Survey on the Robustness of Computer Vision Models against Common Corruptions](https://mlsanity.com/paper/2305.06024/) (2023)
- [Domain Generalization via Rationale Invariance](https://mlsanity.com/paper/2308.11158/) (2023)
- [Understanding Hessian Alignment for Domain Generalization](https://mlsanity.com/paper/2308.11778/) (2023)
- [Domain Generalization in Machine Learning Models for Wireless Communications: Concepts, State-of-the-Art, and Open Issues](https://mlsanity.com/paper/2303.08106/) (2023)
- [When Neural Networks Fail to Generalize? A Model Sensitivity Perspective](https://mlsanity.com/paper/2212.00850/) (2022)
- [Bridging Multicalibration and Out-of-distribution Generalization Beyond Covariate Shift](https://mlsanity.com/paper/2406.00661/) (2024)
- [Hacking Task Confounder in Meta-Learning](https://mlsanity.com/paper/2312.05771/) (2023)
- [Evaluating Robustness and Uncertainty of Graph Models Under Structural Distributional Shifts](https://mlsanity.com/paper/2302.13875/) (2023)
- [OOD-Probe: A Neural Interpretation of Out-of-Domain Generalization](https://mlsanity.com/paper/2208.12352/) (2022)
- [On the Connection between Invariant Learning and Adversarial Training for Out-of-Distribution Generalization](https://mlsanity.com/paper/2212.09082/) (2022)
- [An Empirical Study of Pre-trained Model Selection for Out-of-Distribution Generalization and Calibration](https://mlsanity.com/paper/2307.08187/) (2023)
- [Causality-oriented robustness: exploiting general noise interventions](https://mlsanity.com/paper/2307.10299/) (2023)
- [Towards the Causal Complete Cause of Multi-Modal Representation Learning](https://mlsanity.com/paper/2407.14058/) (2024)
- [Regularization Penalty Optimization for Addressing Data Quality Variance in OoD Algorithms](https://mlsanity.com/paper/2206.05749/) (2022)
- [A Causal Inspired Early-Branching Structure for Domain Generalization](https://mlsanity.com/paper/2403.08649/) (2024)
- [Domain Generalization without Excess Empirical Risk](https://mlsanity.com/paper/2308.15856/) (2023)
- [How Does Distribution Matching Help Domain Generalization: An Information-theoretic Analysis](https://mlsanity.com/paper/2406.09745/) (2024)
- [Benchmarking Domain Generalization Algorithms in Computational Pathology](https://mlsanity.com/paper/2409.17063/) (2024)

## Abstract (arXiv)

The goal of Out-of-Distribution (OOD) generalization problem is to train a predictor that generalizes on all environments. Popular approaches in this field use the hypothesis that such a predictor shall be an textitinvariant predictor that captures the mechanism that remains constant across environments. While these approaches have been experimentally successful in various case studies, there is still much room for the theoretical validation of this hypothesis. This paper presents a new set of theoretical conditions necessary for an invariant predictor to achieve the OOD optimality. Our theory not only applies to non-linear cases, but also generalizes the necessary condition used in citetrojas2018invariant. We also derive Inter Gradient Alignment algorithm from our theory and demonstrate its competitiveness on MNIST-derived benchmark datasets as well as on two of the three textitInvariance Unit Tests proposed by citetaubinlinear.