mlsanityData to 14 Oct 2026

Breaking Correlation Shift via Conditional Invariant Regularizer

arXiv 2207.06687 ↗First posted 14 Jul 2022cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Recently, generalization on out-of-distribution (OOD) data with correlation shift has attracted great attentions. The correlation shift is caused by the spurious attributes that correlate to the class label, as the correlation between them may vary in training and test data. For such a problem, we show that given the class label, the models that are conditionally independent of spurious attributes are OOD generalizable. Based on this, a metric Conditional Spurious Variation (CSV) which controls the OOD generalization error, is proposed to measure such conditional independence. To improve the OOD generalization, we regularize the training process with the proposed CSV. Under mild assumptions, our training objective can be formulated as a nonconvex-concave mini-max problem. An algorithm with a provable convergence rate is proposed to solve the problem. Extensive empirical results verify our algorithm's efficacy in improving OOD generalization.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 48 of its 72 references that are in the map. First posted on arXiv 14 Jul 2022.

Builds on

48 of 72 references.

References · 48

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  4. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  5. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  6. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  7. 2019Invariant Risk Minimizationcited by 1,854
  8. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,515
  9. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,432
  10. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,247
  11. 2020In Search of Lost Domain Generalizationcited by 1,054
  12. 2017Deeper, Broader and Artier Domain Generalizationcited by 986
36 more
  1. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 742
  2. 2013Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programmingcited by 724
  3. 2018Annotation Artifacts in Natural Language Inference Datacited by 671
  4. 2018Recognition in Terra Incognitacited by 658
  5. 2017The Implicit Bias of Gradient Descent on Separable Datacited by 655
  6. 2013Domain Generalization via Invariant Feature Representationcited by 623
  7. 2021Just Train Twice: Improving Group Robustness without Training Group Informationcited by 523
  8. 2018Generalizing to Unseen Domains via Adversarial Data Augmentationcited by 451
  9. 2020Improving robustness against common corruptions by covariate shift adaptationcited by 431
  10. 2019Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classificationcited by 404
  11. 2017Certifying Some Distributional Robustness with Principled Adversarial Trainingcited by 337
  12. 2020Environment Inference for Invariant Learningcited by 306
  13. 2017Information-theoretic analysis of generalization capability of learning algorithmscited by 303
  14. 2019On Gradient Descent Ascent for Nonconvex-Concave Minimax Problemscited by 270
  15. 2020Domain Generalization using Causal Matchingcited by 225
  16. 2020No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problemscited by 209
  17. 2020Large-Scale Methods for Distributionally Robust Optimizationcited by 179
  18. 2021Simple data balancing achieves competitive worst-group-accuracycited by 178
  19. 2020Reasoning About Generalization via Conditional Mutual Informationcited by 147
  20. 2020An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Modelscited by 134
  21. 2021On Calibration and Out-of-domain Generalizationcited by 134
  22. 2021Towards a Theoretical Framework of Out-of-Distribution Generalizationcited by 94
  23. 2021Heterogeneous Risk Minimizationcited by 90
  24. 2021Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Testscited by 77
  25. 2019Domain Generalization via Multidomain Discriminant Analysiscited by 69
  26. 2021Improved OOD Generalization via Adversarial Training and Pre-trainingcited by 66
  27. 2018Marginal Singularity, and the Benefits of Labels in Covariate-Shiftcited by 65
  28. 2020In-N-Out: Pre-Training and Self-Training using Auxiliary Information for Out-of-Distribution Robustnesscited by 50
  29. 2020Unadversarial Examples: Designing Objects for Robust Visioncited by 45
  30. 2022Out-of-distribution Generalization with Causal Invariant Transformationscited by 45
  31. 2014Maximin effects in inhomogeneous large-scale datacited by 32
  32. 2021Reweighting Augmented Samples by Minimizing the Maximal Expected Losscited by 21
  33. 2022Information-Theoretic Bias Reduction via Causal View of Spurious Correlationcited by 15
  34. 2020Optimal Approximation -- Smoothness Tradeoffs for Soft-Max Functionscited by 10
  35. 2020Estimating the Brittleness of AI: Safety Integrity Levels and the Need for Testing Out-Of-Distribution Performancecited by 8
  36. 2019Predicting with High Correlation Featurescited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 2; 2024: 0; 2025: 1; 2026: 2.

7 papers.

  1. 2023On the Generalization of Diffusion Modelcited by 30
  2. 2023Explore and Exploit the Diverse Knowledge in Model Zoo for Domain Generalizationcited by 14
  3. 2022Learning Counterfactually Invariant Predictorscited by 10
  4. 2022Beyond Invariance: Test-Time Label-Shift Adaptation for Distributions with "Spurious" Correlationscited by 7
  5. 2025Improved Diffusion-based Generative Model with Better Adversarial Robustnesscited by 3
  6. 2026TANDEM: Bi-Level Data Mixture Optimization with Twin Networkscited by 3
  7. 2026Fragile Reconstruction: Adversarial Vulnerability of Reconstruction-Based Detectors for Diffusion-Generated Imagescited by 1