# Meta-Learned Invariant Risk Minimization

arXiv: https://arxiv.org/abs/2103.12947 · HTML: https://mlsanity.com/paper/2103.12947/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 27 references that are in the map.
- First posted on arXiv 24 Mar 2021.

## Builds on (most cited first, 20 of 20)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Recognition in Terra Incognita](https://mlsanity.com/paper/1807.04975/) (2018)
- [Causal inference using invariant prediction: identification and confidence intervals](https://mlsanity.com/paper/1501.01332/) (2015)
- [Meta-Learning with Differentiable Convex Optimization](https://mlsanity.com/paper/1904.03758/) (2019)
- [Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML](https://mlsanity.com/paper/1909.09157/) (2019)
- [Insights on representational similarity in neural networks with canonical correlation](https://mlsanity.com/paper/1806.05759/) (2018)
- [Causal Confusion in Imitation Learning](https://mlsanity.com/paper/1905.11979/) (2019)
- [The Risks of Invariant Risk Minimization](https://mlsanity.com/paper/2010.05761/) (2020)
- [Invariant Risk Minimization Games](https://mlsanity.com/paper/2002.04692/) (2020)
- [Invariant Causal Prediction for Nonlinear Models](https://mlsanity.com/paper/1706.08576/) (2017)
- [Does Invariant Risk Minimization Capture Invariance?](https://mlsanity.com/paper/2101.01134/) (2021)
- [Empirical or Invariant Risk Minimization? A Sample Complexity Perspective](https://mlsanity.com/paper/2010.16412/) (2020)
- [Deep Bilevel Learning](https://mlsanity.com/paper/1809.01465/) (2018)
- [Learning to Balance: Bayesian Meta-Learning for Imbalanced and Out-of-distribution Tasks](https://mlsanity.com/paper/1905.12917/) (2019)
- [An Empirical Study of Invariant Risk Minimization](https://mlsanity.com/paper/2004.05007/) (2020)
- [Regularizing Meta-Learning via Gradient Dropout](https://mlsanity.com/paper/2004.05859/) (2020)

## Built on by (most cited first, 7 of 7)

- [Rich Feature Construction for the Optimization-Generalization Dilemma](https://mlsanity.com/paper/2203.15516/) (2022)
- [Optimization-based Causal Estimation from Heterogenous Environments](https://mlsanity.com/paper/2109.11990/) (2021)
- [The Causal Structure of Domain Invariant Supervised Representation Learning](https://mlsanity.com/paper/2208.06987/) (2022)
- [MetaCoCo: A New Few-Shot Classification Benchmark with Spurious Correlation](https://mlsanity.com/paper/2404.19644/) (2024)
- [Causal Effect Identification in Heterogeneous Environments from Higher-Order Moments](https://mlsanity.com/paper/2506.11756/) (2025)
- [Invariance Pair Guidance: Robustness to Spurious Correlations via Corrective Gradients](https://mlsanity.com/paper/2502.18975/) (2025)
- [Universal Adaptive Environment Discovery](https://mlsanity.com/paper/2510.12547/) (2025)

## Abstract (arXiv)

Empirical Risk Minimization (ERM) based machine learning algorithms have suffered from weak generalization performance on data obtained from out-of-distribution (OOD). To address this problem, Invariant Risk Minimization (IRM) objective was suggested to find invariant optimal predictor which is less affected by the changes in data distribution. However, even with such progress, IRMv1, the practical formulation of IRM, still shows performance degradation when there are not enough training data, and even fails to generalize to OOD, if the number of spurious correlations is larger than the number of environments. In this paper, to address such problems, we propose a novel meta-learning based approach for IRM. In this method, we do not assume the linearity of classifier for the ease of optimization, and solve ideal bi-level IRM objective with Model-Agnostic Meta-Learning (MAML) framework. Our method is more robust to the data with spurious correlations and can provide an invariant optimal classifier even when data from each distribution are scarce. In experiments, we demonstrate that our algorithm not only has better OOD generalization performance than IRMv1 and all IRM variants, but also addresses the weakness of IRMv1 with improved stability.