mlsanityData to 7 Oct 2026

On the Connection between Invariant Learning and Adversarial Training for Out-of-Distribution Generalization

arXiv 2212.09082 ↗First posted 18 Dec 2022cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Despite impressive success in many tasks, deep learning models are shown to rely on spurious features, which will catastrophically fail when generalized to out-of-distribution (OOD) data. Invariant Risk Minimization (IRM) is proposed to alleviate this issue by extracting domain-invariant features for OOD generalization. Nevertheless, recent work shows that IRM is only effective for a certain type of distribution shift (e.g., correlation shift) while it fails for other cases (e.g., diversity shift). Meanwhile, another thread of method, Adversarial Training (AT), has shown better domain transfer performance, suggesting that it has the potential to be an effective candidate for extracting domain-invariant features. This paper investigates this possibility by exploring the similarity between the IRM and AT objectives. Inspired by this connection, we propose Domainwise Adversarial Training (DAT), an AT-inspired method for alleviating distribution shift by domain-specific perturbations. Extensive experiments show that our proposed DAT can effectively remove domain-varying features and improve OOD generalization under both correlation shift and diversity shift.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 45 of its 59 references that are in the map. First posted on arXiv 18 Dec 2022.

Builds on

45 of 59 references.

References · 45

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  6. 2013Intriguing properties of neural networkscited by 5,114
  7. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  8. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,906
  9. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  10. 2019Invariant Risk Minimizationcited by 1,832
  11. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  12. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,194
33 more
  1. 2020In Search of Lost Domain Generalizationcited by 1,042
  2. 2017Deeper, Broader and Artier Domain Generalizationcited by 983
  3. 2016Universal adversarial perturbationscited by 774
  4. 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 762
  5. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 737
  6. 2018Recognition in Terra Incognitacited by 655
  7. 2015Causal inference using invariant prediction: identification and confidence intervalscited by 533
  8. 2018Generalizing to Unseen Domains via Adversarial Data Augmentationcited by 451
  9. 2020Self-Challenging Improves Cross-Domain Generalizationcited by 426
  10. 2020Noise or Signal: The Role of Image Backgrounds in Object Recognitioncited by 342
  11. 2017Certifying Some Distributional Robustness with Principled Adversarial Trainingcited by 333
  12. 2020Do Adversarially Robust ImageNet Models Transfer Better?cited by 315
  13. 2020Learning to Learn Single Domain Generalizationcited by 295
  14. 2018Generalizing Across Domains via Cross-Gradient Trainingcited by 268
  15. 2020The Risks of Invariant Risk Minimizationcited by 262
  16. 2020Invariant Risk Minimization Gamescited by 188
  17. 2020Improve Unsupervised Domain Adaptation with Mixup Trainingcited by 185
  18. 2020Adaptive Risk Minimization: Learning to Adapt to Domain Shiftcited by 183
  19. 2016Statistics of Robust Optimization: A Generalized Empirical Likelihood Approachcited by 182
  20. 2017Domain Generalization by Marginal Transfer Learningcited by 180
  21. 2021On the Convergence and Robustness of Adversarial Trainingcited by 127
  22. 2021Does Invariant Risk Minimization Capture Invariance?cited by 116
  23. 2019Adversarial Examples Are a Natural Consequence of Test Error in Noisecited by 114
  24. 2021OoD-Bench: Quantifying and Understanding Two Dimensions of Out-of-Distribution Generalizationcited by 86
  25. 2018Universal Adversarial Trainingcited by 85
  26. 2021Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?cited by 75
  27. 2021Improved OOD Generalization via Adversarial Training and Pre-trainingcited by 66
  28. 2020When is invariance useful in an Out-of-Distribution Generalization problem ?cited by 52
  29. 2022Self-Ensemble Adversarial Training for Improved Robustnesscited by 38
  30. 2020Attribute-Guided Adversarial Training for Robustness to Natural Perturbationscited by 23
  31. 2017Residual Convolutional CTC Networks for Automatic Speech Recognitioncited by 21
  32. 2021A Unified Game-Theoretic Interpretation of Adversarial Robustnesscited by 17
  33. 2022Improving Out-of-Distribution Generalization by Adversarial Training with Structured Priorscited by 17

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 2; 2025: 2.

5 papers.

  1. 2023Defenses in Adversarial Machine Learning: A Surveycited by 14
  2. 2024Beyond Interpretability: The Gains of Feature Monosemanticity on Model Robustnesscited by 13
  3. 2025Out-of-distribution Generalization for Total Variation based Invariant Risk Minimizationcited by 3
  4. 2024Combining Machine Learning Defenses without Conflictscited by 2
  5. 2025Adversarial Training Improves Generalization Under Distribution Shifts in Bird Sound Classificationcited by 0