mlsanityData to 13 Oct 2026

The Two Dimensions of Worst-case Training and the Integrated Effect for Out-of-domain Generalization

arXiv 2204.04384 ↗First posted 9 Apr 2022cs.LG cs.CVcited by 21

Not on the map. Why?

Abstract (arXiv)

Training with an emphasis on "hard-to-learn" components of the data has been proven as an effective method to improve the generalization of machine learning models, especially in the settings where robustness (e.g., generalization across distributions) is valued. Existing literature discussing this "hard-to-learn" concept are mainly expanded either along the dimension of the samples or the dimension of the features. In this paper, we aim to introduce a simple view merging these two dimensions, leading to a new, simple yet effective, heuristic to train machine learning models by emphasizing the worst-cases on both the sample and the feature dimensions. We name our method W2D following the concept of "Worst-case along Two Dimensions". We validate the idea and demonstrate its empirical strength over standard benchmarks.

Key facts. Cited by 21 papers in the map (data through 13 Oct 2026). Builds on 54 of its 80 references that are in the map. First posted on arXiv 9 Apr 2022.

Builds on

54 of 80 references.

References · 54

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,950
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,336
  3. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,170
  4. 2013Intriguing properties of neural networkscited by 5,137
  5. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,055
  6. 2014Deep Learning Face Attributes in the Wildcited by 3,686
  7. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,601
  8. 2015Domain-Adversarial Training of Neural Networkscited by 3,030
  9. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,862
  10. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,915
  11. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,847
  12. 2019Invariant Risk Minimizationcited by 1,842
38 more
  1. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  2. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,371
  3. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,242
  4. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,202
  5. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,196
  6. 2017Deep Hashing Network for Unsupervised Domain Adaptationcited by 1,183
  7. 2020In Search of Lost Domain Generalizationcited by 1,048
  8. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,012
  9. 2017Deeper, Broader and Artier Domain Generalizationcited by 984
  10. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 899
  11. 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 763
  12. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 738
  13. 2018Recognition in Terra Incognitacited by 656
  14. 2013Domain Generalization via Invariant Feature Representationcited by 623
  15. 2020Self-Challenging Improves Cross-Domain Generalizationcited by 426
  16. 2019Bridging Theory and Algorithm for Domain Adaptationcited by 332
  17. 2018Not All Samples Are Created Equal: Deep Learning with Importance Samplingcited by 327
  18. 2017Unified Deep Supervised Domain Adaptation and Generalizationcited by 323
  19. 2019High Frequency Component Helps Explain the Generalization of Convolutional Neural Networkscited by 310
  20. 2015Domain Generalization for Object Recognition with Multi-task Autoencoderscited by 253
  21. 2020Fairness without Demographics through Adversarially Reweighted Learningcited by 251
  22. 2018What is the Effect of Importance Weighting in Deep Learning?cited by 250
  23. 2019Learning De-biased Representations with Biased Representationscited by 199
  24. 2017Domain Generalization by Marginal Transfer Learningcited by 180
  25. 2020Learning Texture Invariant Representation for Domain Adaptation of Semantic Segmentationcited by 153
  26. 2017Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samplescited by 147
  27. 2020Learning from Failure: Training Debiased Classifier from Biased Classifiercited by 133
  28. 2020Learning explanations that are hard to varycited by 132
  29. 2019Learning Robust Representations by Projecting Superficial Statistics Outcited by 123
  30. 2018There Are Many Consistent Explanations of Unlabeled Data: Why You Should Averagecited by 82
  31. 2019Domain Adaptation with Asymmetrically-Relaxed Distribution Alignmentcited by 61
  32. 2021Domain Invariant Representation Learning with Domain Density Transformationscited by 53
  33. 2019Adversarial Invariant Feature Learning with Accuracy Constraint for Domain Generalizationcited by 43
  34. 2019Reducing Domain Gap by Reducing Style Biascited by 35
  35. 2015A New PAC-Bayesian Perspective on Domain Adaptationcited by 29
  36. 2016Select-Additive Learning: Improving Generalization in Multimodal Sentiment Analysiscited by 29
  37. 2021Toward Learning Human-aligned Cross-domain Robust Models by Countering Misaligned Featurescited by 14
  38. 2021Learning Domain Invariant Representations in Goal-conditioned Block MDPscited by 12

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2022: 3; 2023: 7; 2024: 7; 2025: 3; 2026: 1.

21 papers.

Sort:

By area:

  1. 2023A Sentence Speaks a Thousand Images: Domain Generalization through Distilling CLIP with Language Guidancecited by 40
  2. 2023ALOFT: A Lightweight MLP-like Architecture with Dynamic Low-frequency Transform for Domain Generalizationcited by 28
  3. 2024CreINNs: Credal-Set Interval Neural Networks for Uncertainty Estimation in Classification Taskscited by 26
  4. 2023DomainDrop: Suppressing Domain-Sensitive Channels for Domain Generalizationcited by 21
  5. 2023Rethinking the Evaluation Protocol of Domain Generalizationcited by 20
  6. 2023Domain Generalization Guided by Gradient Signal to Noise Ratio of Parameterscited by 9
  7. 2024LFME: A Simple Framework for Learning from Multiple Experts in Domain Generalizationcited by 8
  8. 2026Learning Credal Ensembles via Distributionally Robust Optimizationcited by 7
  9. 2022OOD-Probe: A Neural Interpretation of Out-of-Domain Generalizationcited by 5
  10. 2024Beyond Finite Data: Towards Data-free Out-of-distribution Generalization via Extrapolationcited by 4
  11. 2024CHG Shapley: Efficient Data Valuation and Selection towards Trustworthy Machine Learningcited by 4
  12. 2024Decision-Focused Evaluation of Worst-Case Distribution Shiftcited by 4
9 more
  1. 2024UnLearning from Experience to Avoid Spurious Correlationscited by 2
  2. 2025Credal Graph Neural Networkscited by 2
  3. 2025Robust Context-Aware Object Recognitioncited by 1
  4. 2022Robustar: Interactive Toolbox Supporting Precise Data Annotation for Robust Vision Learningcited by 0
  5. 2022Expeditious Saliency-guided Mix-up through Random Gradient Thresholdingcited by 0
  6. 2023Choosing Wisely and Learning Deeply: Selective Cross-Modality Distillation via CLIP for Domain Generalizationcited by 0
  7. 2023Domain Invariant Learning for Gaussian Processes and Bayesian Explorationcited by 0
  8. 2024Bringing the Context Back into Object Recognition, Robustlycited by 0
  9. 2025FLIP: Towards Comprehensive and Reliable Evaluation of Federated Prompt Learningcited by 0