mlsanityData to 7 Oct 2026

Invariant Correlation of Representation with Label: Enhancing Domain Generalization in Noisy Environments

arXiv 2407.01749 ↗First posted 1 Jul 2024cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

The Invariant Risk Minimization (IRM) approach aims to address the challenge of domain generalization by training a feature representation that remains invariant across multiple environments. However, in noisy environments, IRM-related techniques such as IRMv1 and VREx may be unable to achieve the optimal IRM solution, primarily due to erroneous optimization directions. To address this issue, we introduce ICorr (an abbreviation for Invariant Correlation), a novel approach designed to surmount the above challenge in noisy settings. Additionally, we dig into a case study to analyze why previous methods may lose ground while ICorr can succeed. Through a theoretical lens, particularly from a causality perspective, we illustrate that the invariant correlation of representation with label is a necessary condition for the optimal invariant predictor in noisy environments, whereas the optimization motivations for other methods may not be. Furthermore, we empirically demonstrate the effectiveness of ICorr by comparing it with other domain generalization methods on various noisy datasets. The code is available at https://github.com/Alexkael/ICorr.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 74 of its 109 references that are in the map. First posted on arXiv 1 Jul 2024.

Builds on

74 of 109 references.

References · 74

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  4. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  5. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  6. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  7. 2019Invariant Risk Minimizationcited by 1,832
  8. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
  9. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,194
  10. 2020In Search of Lost Domain Generalizationcited by 1,042
  11. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,024
  12. 2017Deeper, Broader and Artier Domain Generalizationcited by 983
38 more
  1. 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 762
  2. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 737
  3. 2018Recognition in Terra Incognitacited by 655
  4. 2013Domain Generalization via Invariant Feature Representationcited by 622
  5. 2015Causal inference using invariant prediction: identification and confidence intervalscited by 533
  6. 2021Just Train Twice: Improving Group Robustness without Training Group Informationcited by 522
  7. 2018Generalizing to Unseen Domains via Adversarial Data Augmentationcited by 451
  8. 2019Domain Generalization by Solving Jigsaw Puzzlescited by 449
  9. 2021SWAD: Domain Generalization by Seeking Flat Minimacited by 431
  10. 2017Causal Effect Inference with Deep Latent-Variable Modelscited by 408
  11. 2019Domain Generalization via Model-Agnostic Learning of Semantic Featurescited by 405
  12. 2017Unified Deep Supervised Domain Adaptation and Generalizationcited by 322
  13. 2020Environment Inference for Invariant Learningcited by 306
  14. 2020Learning to Generate Novel Domains for Domain Generalizationcited by 296
  15. 2021A Fourier-based Framework for Domain Generalizationcited by 287
  16. 2018Generalizing Across Domains via Cross-Gradient Trainingcited by 268
  17. 2020The Risks of Invariant Risk Minimizationcited by 262
  18. 2021Gradient Matching for Domain Generalizationcited by 258
  19. 2019Episodic Training for Domain Generalizationcited by 245
  20. 2021Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalizationcited by 220
  21. 2022Improving Out-of-Distribution Robustness via Selective Augmentationcited by 202
  22. 2020Representation Learning via Invariant Causal Mechanismscited by 197
  23. 2020Invariant Risk Minimization Gamescited by 188
  24. 2021Fishr: Invariant Gradient Variances for Out-of-Distribution Generalizationcited by 186
  25. 2022Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlationscited by 171
  26. 2021Ensemble of Averages: Improving Model Selection and Boosting Performance in Domain Generalizationcited by 144
  27. 2022Causality Inspired Representation Learning for Domain Generalizationcited by 136
  28. 2021On Calibration and Out-of-domain Generalizationcited by 134
  29. 2020Learning explanations that are hard to varycited by 132
  30. 2019Feature-Critic Networks for Heterogeneous Domain Generalizationcited by 130
  31. 2021Does Invariant Risk Minimization Capture Invariance?cited by 116
  32. 2019Learning Neural Causal Models from Unknown Interventionscited by 113
  33. 2020Learning from Extrinsic and Intrinsic Supervisions for Domain Generalizationcited by 108
  34. 2021Model-Based Domain Generalizationcited by 103
  35. 2020Continuously Indexed Domain Adaptationcited by 74
  36. 2020Empirical or Invariant Risk Minimization? A Sample Complexity Perspectivecited by 62
  37. 2021Domain Generalization via Gradient Surgerycited by 60
  38. 2021SAND-mask: An Enhanced Gradient Masking Strategy for the Discovery of Invariances in Domain Generalizationcited by 50

50 most cited of 74 All 74 (JSON)

Built on by

Not cited yet.