mlsanityData to 7 Oct 2026

The Missing Invariance Principle Found -- the Reciprocal Twin of Invariant Risk Minimization

arXiv 2205.14546 ↗First posted 29 May 2022cs.LG cs.AI cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

Machine learning models often generalize poorly to out-of-distribution (OOD) data as a result of relying on features that are spuriously correlated with the label during training. Recently, the technique of Invariant Risk Minimization (IRM) was proposed to learn predictors that only use invariant features by conserving the feature-conditioned label expectation mathbbEe[y|f(x)] across environments. However, more recent studies have demonstrated that IRM-v1, a practical version of IRM, can fail in various settings. Here, we identify a fundamental flaw of IRM formulation that causes the failure. We then introduce a complementary notion of invariance, MRI, based on conserving the label-conditioned feature expectation mathbbEe[f(x)|y], which is free of this flaw. Further, we introduce a simplified, practical version of the MRI formulation called MRI-v1. We prove that for general linear problems, MRI-v1 guarantees invariant predictors given sufficient number of environments. We also empirically demonstrate that MRI-v1 strongly out-performs IRM-v1 and consistently achieves near-optimal OOD generalization in image-based nonlinear problems.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 27 of its 38 references that are in the map. First posted on arXiv 29 May 2022.

Builds on

27 of 38 references.

References · 27

  1. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  2. 2019Invariant Risk Minimizationcited by 1,832
  3. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
  4. 2020In Search of Lost Domain Generalizationcited by 1,042
  5. 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 762
  6. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 737
  7. 2018Recognition in Terra Incognitacited by 655
  8. 2013Domain Generalization via Invariant Feature Representationcited by 622
  9. 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 585
  10. 2015Causal inference using invariant prediction: identification and confidence intervalscited by 533
  11. 2018Essentially No Barriers in Neural Network Energy Landscapecited by 328
  12. 2020Environment Inference for Invariant Learningcited by 306
15 more
  1. 2018Generalizing Across Domains via Cross-Gradient Trainingcited by 268
  2. 2020The Risks of Invariant Risk Minimizationcited by 262
  3. 2020Domain Generalization using Causal Matchingcited by 224
  4. 2021Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalizationcited by 220
  5. 2021On the Origin of Implicit Regularization in Stochastic Gradient Descentcited by 183
  6. 2018Domain Generalization via Conditional Invariant Representationcited by 146
  7. 2017Invariant Causal Prediction for Nonlinear Modelscited by 140
  8. 2021On Calibration and Out-of-domain Generalizationcited by 134
  9. 2021Does Invariant Risk Minimization Capture Invariance?cited by 116
  10. 2019DIVA: Domain Invariant Variational Autoencoderscited by 114
  11. 2019Generalizing to unseen domains via distribution matchingcited by 109
  12. 2015Scatter Component Analysis: A Unified Framework for Domain Adaptation and Domain Generalizationcited by 96
  13. 2019Domain Generalization via Multidomain Discriminant Analysiscited by 69
  14. 2020Empirical or Invariant Risk Minimization? A Sample Complexity Perspectivecited by 62
  15. 2019Adversarial Invariant Feature Learning with Accuracy Constraint for Domain Generalizationcited by 43

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 2; 2024: 2; 2025: 0; 2026: 3.

8 papers.

  1. 2022Domain Generalization -- A Causal Perspectivecited by 14
  2. 2024Bridging Multicalibration and Out-of-distribution Generalization Beyond Covariate Shiftcited by 10
  3. 2024Towards Understanding Variants of Invariant Risk Minimization through the Lens of Calibrationcited by 3
  4. 2026Distribution Shift Is Key to Learning Invariant Predictioncited by 2
  5. 2023Out-of-Distribution Optimality of Invariant Risk Minimizationcited by 1
  6. 2023Diagnosing and Rectifying Fake OOD Invariance: A Restructured Causal Approachcited by 1
  7. 2026CORE-MTL: Rethinking Gradient Balancing via Causal Orthogonal Representationscited by 0
  8. 2026Conditionally Identifiable Latent-Environment Modeling for Out-of-Distribution Recommendationcited by 0