# The Missing Invariance Principle Found -- the Reciprocal Twin of Invariant Risk Minimization

arXiv: https://arxiv.org/abs/2205.14546 · HTML: https://mlsanity.com/paper/2205.14546/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 38 references that are in the map.
- First posted on arXiv 29 May 2022.

## Builds on (most cited first, 27 of 27)

- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Recognition in Terra Incognita](https://mlsanity.com/paper/1807.04975/) (2018)
- [Domain Generalization via Invariant Feature Representation](https://mlsanity.com/paper/1301.2115/) (2013)
- [Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs](https://mlsanity.com/paper/1802.10026/) (2018)
- [Causal inference using invariant prediction: identification and confidence intervals](https://mlsanity.com/paper/1501.01332/) (2015)
- [Essentially No Barriers in Neural Network Energy Landscape](https://mlsanity.com/paper/1803.00885/) (2018)
- [Environment Inference for Invariant Learning](https://mlsanity.com/paper/2010.07249/) (2020)
- [Generalizing Across Domains via Cross-Gradient Training](https://mlsanity.com/paper/1804.10745/) (2018)
- [The Risks of Invariant Risk Minimization](https://mlsanity.com/paper/2010.05761/) (2020)
- [Domain Generalization using Causal Matching](https://mlsanity.com/paper/2006.07500/) (2020)
- [Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization](https://mlsanity.com/paper/2106.06607/) (2021)
- [On the Origin of Implicit Regularization in Stochastic Gradient Descent](https://mlsanity.com/paper/2101.12176/) (2021)
- [Domain Generalization via Conditional Invariant Representation](https://mlsanity.com/paper/1807.08479/) (2018)
- [Invariant Causal Prediction for Nonlinear Models](https://mlsanity.com/paper/1706.08576/) (2017)
- [On Calibration and Out-of-domain Generalization](https://mlsanity.com/paper/2102.10395/) (2021)
- [Does Invariant Risk Minimization Capture Invariance?](https://mlsanity.com/paper/2101.01134/) (2021)
- [DIVA: Domain Invariant Variational Autoencoders](https://mlsanity.com/paper/1905.10427/) (2019)
- [Generalizing to unseen domains via distribution matching](https://mlsanity.com/paper/1911.00804/) (2019)
- [Scatter Component Analysis: A Unified Framework for Domain Adaptation and Domain Generalization](https://mlsanity.com/paper/1510.04373/) (2015)
- [Domain Generalization via Multidomain Discriminant Analysis](https://mlsanity.com/paper/1907.11216/) (2019)
- [Empirical or Invariant Risk Minimization? A Sample Complexity Perspective](https://mlsanity.com/paper/2010.16412/) (2020)
- [Adversarial Invariant Feature Learning with Accuracy Constraint for Domain Generalization](https://mlsanity.com/paper/1904.12543/) (2019)

## Built on by (most cited first, 8 of 8)

- [Domain Generalization -- A Causal Perspective](https://mlsanity.com/paper/2209.15177/) (2022)
- [Bridging Multicalibration and Out-of-distribution Generalization Beyond Covariate Shift](https://mlsanity.com/paper/2406.00661/) (2024)
- [Towards Understanding Variants of Invariant Risk Minimization through the Lens of Calibration](https://mlsanity.com/paper/2401.17541/) (2024)
- [Distribution Shift Is Key to Learning Invariant Prediction](https://mlsanity.com/paper/2601.12296/) (2026)
- [Out-of-Distribution Optimality of Invariant Risk Minimization](https://mlsanity.com/paper/2307.11972/) (2023)
- [Diagnosing and Rectifying Fake OOD Invariance: A Restructured Causal Approach](https://mlsanity.com/paper/2312.09758/) (2023)
- [CORE-MTL: Rethinking Gradient Balancing via Causal Orthogonal Representations](https://mlsanity.com/paper/2606.02221/) (2026)
- [Conditionally Identifiable Latent-Environment Modeling for Out-of-Distribution Recommendation](https://mlsanity.com/paper/2608.03647/) (2026)

## Abstract (arXiv)

Machine learning models often generalize poorly to out-of-distribution (OOD) data as a result of relying on features that are spuriously correlated with the label during training. Recently, the technique of Invariant Risk Minimization (IRM) was proposed to learn predictors that only use invariant features by conserving the feature-conditioned label expectation mathbbEe[y|f(x)] across environments. However, more recent studies have demonstrated that IRM-v1, a practical version of IRM, can fail in various settings. Here, we identify a fundamental flaw of IRM formulation that causes the failure. We then introduce a complementary notion of invariance, MRI, based on conserving the label-conditioned feature expectation mathbbEe[f(x)|y], which is free of this flaw. Further, we introduce a simplified, practical version of the MRI formulation called MRI-v1. We prove that for general linear problems, MRI-v1 guarantees invariant predictors given sufficient number of environments. We also empirically demonstrate that MRI-v1 strongly out-performs IRM-v1 and consistently achieves near-optimal OOD generalization in image-based nonlinear problems.