mlsanityData to 7 Oct 2026

Does Invariant Risk Minimization Capture Invariance?

arXiv 2101.01134 ↗First posted 4 Jan 2021stat.ML cs.AI cs.LGcited by 116

Not on the map. Why?

Abstract (arXiv)

We show that the Invariant Risk Minimization (IRM) formulation of Arjovsky et al. (2019) can fail to capture "natural" invariances, at least when used in its practical "linear" form, and even on very simple problems which directly follow the motivating examples for IRM. This can lead to worse generalization on new environments, even when compared to unconstrained ERM. The issue stems from a significant gap between the linear variant (as in their concrete method IRMv1) and the full non-linear IRM formulation. Additionally, even when capturing the "right" invariances, we show that it is possible for IRM to learn a sub-optimal predictor, due to the loss function not being invariant across environments. The issues arise even when measuring invariance on the population distributions, but are exacerbated by the fact that IRM is extremely fragile to sampling.

Key facts. Cited by 116 papers in the map (data through 7 Oct 2026). Builds on 12 of its 21 references that are in the map. First posted on arXiv 4 Jan 2021.

Builds on

12 of 21 references.

References · 12

  1. 2019Invariant Risk Minimizationcited by 1,832
  2. 2020In Search of Lost Domain Generalizationcited by 1,042
  3. 2018Recognition in Terra Incognitacited by 655
  4. 2015Causal inference using invariant prediction: identification and confidence intervalscited by 533
  5. 2020The Risks of Invariant Risk Minimizationcited by 262
  6. 2020Invariant Risk Minimization Gamescited by 188
  7. 2020Understanding the Failure Modes of Out-of-Distribution Generalizationcited by 146
  8. 2017Invariant Causal Prediction for Nonlinear Modelscited by 140
  9. 2020Invariant Rationalizationcited by 131
  10. 2020A survey on domain adaptation theory: learning bounds and theoretical guaranteescited by 109
  11. 2020Empirical or Invariant Risk Minimization? A Sample Complexity Perspectivecited by 62
  12. 2020Unshuffling Data for Improved Generalizationcited by 35

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 25; 2022: 30; 2023: 23; 2024: 16; 2025: 15; 2026: 7.

116 papers.

Sort:

By area:

  1. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 722
  2. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 431
  3. 2021Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalizationcited by 220
  4. 2021Fishr: Invariant Gradient Variances for Out-of-Distribution Generalizationcited by 186
  5. 2022Discovering Invariant Rationales for Graph Neural Networkscited by 186
  6. 2022Handling Distribution Shifts on Graphs: An Invariance Perspectivecited by 163
  7. 2021On Calibration and Out-of-domain Generalizationcited by 134
  8. 2021Model-Based Domain Generalizationcited by 103
  9. 2021Towards a Theoretical Framework of Out-of-Distribution Generalizationcited by 94
  10. 2021Towards Principled Disentanglement for Domain Generalizationcited by 91
  11. 2021Invariant Information Bottleneck for Domain Generalizationcited by 86
  12. 2021Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?cited by 75
38 more
  1. 2021Generalizing Graph Neural Networks on Out-Of-Distribution Graphscited by 69
  2. 2022Domain-Adjusted Regression or: ERM May Already Learn Features Sufficient for Out-of-Distribution Generalizationcited by 69
  3. 2022Causal Discovery in Heterogeneous Environments Under the Sparse Mechanism Shift Hypothesiscited by 54
  4. 2023Rethinking Domain Generalization for Face Anti-spoofing: Separability and Alignmentcited by 47
  5. 2022Out-of-distribution Generalization with Causal Invariant Transformationscited by 45
  6. 2021Nonlinear Invariant Risk Minimization: A Causal Approachcited by 42
  7. 2023Does Invariant Graph Learning via Environment Augmentation Learn Invariance?cited by 42
  8. 2023Spawrious: A Benchmark for Fine Control of Spurious Correlation Biasescited by 41
  9. 2021Near-Optimal Linear Regression under Distribution Shiftcited by 40
  10. 2023An Introduction to Bi-level Optimization: Foundations and Applications in Signal Processing and Machine Learningcited by 39
  11. 2022Pareto Invariant Risk Minimization: Towards Mitigating the Optimization Dilemma in Out-of-Distribution Generalizationcited by 38
  12. 2022Unleashing the Power of Graph Data Augmentation on Covariate Distribution Shiftcited by 38
  13. 2023Understanding and Improving Feature Learning for Out-of-Distribution Generalizationcited by 35
  14. 2022Rich Feature Construction for the Optimization-Generalization Dilemmacited by 34
  15. 2021An Empirical Framework for Domain Generalization in Clinical Settingscited by 32
  16. 2022Provable Domain Generalization via Invariant-Feature Subspace Recoverycited by 32
  17. 2022Minimax Regret Optimization for Robust Machine Learning under Distribution Shiftcited by 31
  18. 2021Iterative Feature Matching: Toward Provable Domain Generalization with Logarithmic Environmentscited by 30
  19. 2023Towards Trustworthy and Aligned Machine Learning: A Data-centric Survey with Causality Perspectivescited by 30
  20. 2022Debiased Graph Neural Networks with Agnostic Label Selection Biascited by 25
  21. 2025Towards LLMs Robustness to Changes in Prompt Format Stylescited by 21
  22. 2021Regularizing towards Causal Invariance: Linear Models with Proxiescited by 20
  23. 2022Causal Balancing for Domain Generalizationcited by 20
  24. 2021On the benefits of representation regularization in invariance based domain generalizationcited by 17
  25. 2023Environment Invariant Linear Least Squarescited by 17
  26. 2021Optimization-based Causal Estimation from Heterogenous Environmentscited by 16
  27. 2022FL Games: A Federated Learning Framework for Distribution Shiftscited by 14
  28. 2023Data Augmentations for Improved (Large) Language Model Generalizationcited by 14
  29. 2024Domain Generalisation via Imprecise Learningcited by 14
  30. 2021Secure Domain Adaptation with Multiple Sourcescited by 12
  31. 2024Feature contamination: Neural networks learn uncorrelated features and fail to generalizecited by 11
  32. 2021On Invariance Penalties for Risk Minimizationcited by 10
  33. 2021Finding Valid Adjustments under Non-ignorability with Minimal DAG Knowledgecited by 10
  34. 2023What Is Missing in IRM Training and Evaluation? Challenges and Solutionscited by 10
  35. 2024Seeking Consistent Flat Minima for Better Domain Generalization via Refining Loss Landscapescited by 10
  36. 2022The Missing Invariance Principle Found -- the Reciprocal Twin of Invariant Risk Minimizationcited by 8
  37. 2023PGrad: Learning Principal Gradients For Domain Generalizationcited by 8
  38. 2021Meta-Learned Invariant Risk Minimizationcited by 7

50 most cited of 116 All 116 (JSON)