mlsanityData to 14 Oct 2026

Out-of-Distribution Optimality of Invariant Risk Minimization

arXiv 2307.11972 ↗First posted 22 Jul 2023stat.ML cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Deep Neural Networks often inherit spurious correlations embedded in training data and hence may fail to generalize to unseen domains, which have different distributions from the domain to provide training data. M. Arjovsky et al. (2019) introduced the concept out-of-distribution (o.o.d.) risk, which is the maximum risk among all domains, and formulated the issue caused by spurious correlations as a minimization problem of the o.o.d. risk. Invariant Risk Minimization (IRM) is considered to be a promising approach to minimize the o.o.d. risk: IRM estimates a minimum of the o.o.d. risk by solving a bi-level optimization problem. While IRM has attracted considerable attention with empirical success, it comes with few theoretical guarantees. Especially, a solid theoretical guarantee that the bi-level optimization problem gives the minimum of the o.o.d. risk has not yet been established. Aiming at providing a theoretical justification for IRM, this paper rigorously proves that a solution to the bi-level optimization problem minimizes the o.o.d. risk under certain conditions. The result also provides sufficient conditions on distributions providing training data and on a dimension of feature space for the bi-leveled optimization problem to minimize the o.o.d. risk.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 22 of its 36 references that are in the map. First posted on arXiv 22 Jul 2023.

Builds on

22 of 36 references.

References · 22

  1. 2019Invariant Risk Minimizationcited by 1,854
  2. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 742
  3. 2018Recognition in Terra Incognitacited by 658
  4. 2015Causal inference using invariant prediction: identification and confidence intervalscited by 536
  5. 2020The Risks of Invariant Risk Minimizationcited by 262
  6. 2021Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalizationcited by 220
  7. 2020Invariant Risk Minimization Gamescited by 188
  8. 2021Fishr: Invariant Gradient Variances for Out-of-Distribution Generalizationcited by 186
  9. 2020Learning explanations that are hard to varycited by 132
  10. 2020Invariant Rationalizationcited by 131
  11. 2021Does Invariant Risk Minimization Capture Invariance?cited by 116
  12. 2018Unsupervised domain adaptation for medical imaging segmentation with self-ensemblingcited by 94
10 more
  1. 2021Heterogeneous Risk Minimizationcited by 90
  2. 2015Neural Network with Unbounded Activation Functions is Universal Approximatorcited by 63
  3. 2022ZIN: When and How to Learn Invariance Without Environment Partition?cited by 58
  4. 2020When is invariance useful in an Out-of-Distribution Generalization problem ?cited by 52
  5. 2022Pareto Invariant Risk Minimization: Towards Mitigating the Optimization Dilemma in Out-of-Distribution Generalizationcited by 38
  6. 2021Kernelized Heterogeneous Risk Minimizationcited by 21
  7. 2022Efficient Conditionally Invariant Representation Learningcited by 19
  8. 2022The Missing Invariance Principle Found -- the Reciprocal Twin of Invariant Risk Minimizationcited by 8
  9. 2020Linear Regression Games: Convergence Guarantees to Approximate Out-of-Distribution Solutionscited by 7
  10. 2022Invariance Learning based on Label Hierarchycited by 1

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Unsupervised Representation Learning - an Invariant Risk Minimization Perspectivecited by 2