Out-of-distribution Generalization for Total Variation based Invariant Risk Minimization
Not on the map. Why?
Abstract (arXiv)
Invariant risk minimization is an important general machine learning framework that has recently been interpreted as a total variation model (IRM-TV). However, how to improve out-of-distribution (OOD) generalization in the IRM-TV setting remains unsolved. In this paper, we extend IRM-TV to a Lagrangian multiplier model named OOD-TV-IRM. We find that the autonomous TV penalty hyperparameter is exactly the Lagrangian multiplier. Thus OOD-TV-IRM is essentially a primal-dual optimization model, where the primal optimization minimizes the entire invariant risk and the dual optimization strengthens the TV penalty. The objective is to reach a semi-Nash equilibrium where the balance between the training loss and OOD generalization is maintained. We also develop a convergent primal-dual algorithm that facilitates an adversarial learning scheme. Experimental results show that OOD-TV-IRM outperforms IRM-TV in most situations.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 16 of its 27 references that are in the map. First posted on arXiv 27 Feb 2025.
Builds on
16 of 27 references.
References · 16
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2014Deep Learning Face Attributes in the Wildcited by 3,679
- 2019Invariant Risk Minimizationcited by 1,832
- 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
- 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 737
- 2020Environment Inference for Invariant Learningcited by 306
- 2020Invariant Risk Minimization Gamescited by 188
- 2021Heterogeneous Risk Minimizationcited by 90
- 2022ZIN: When and How to Learn Invariance Without Environment Partition?cited by 58
- 2020In-N-Out: Pre-Training and Self-Training using Auxiliary Information for Out-of-Distribution Robustnesscited by 49
- 2022Harnessing Out-Of-Distribution Examples via Augmenting Content and Stylecited by 43
- 2020Meta-Learning for Few-Shot Land Cover Classificationcited by 34
4 more
- 2023Spurious Feature Diversification Improves Out-of-distribution Generalizationcited by 26
- 2023Invariant Learning via Probability of Sufficient and Necessary Causescited by 22
- 2022On the Connection between Invariant Learning and Adversarial Training for Out-of-Distribution Generalizationcited by 5
- 2024Invariant Risk Minimization Is A Total Variation Modelcited by 5
Built on by
Citing papers per year: 2025: 1; 2026: 2.
3 papers.