mlsanityData to 7 Oct 2026

Towards Optimization and Model Selection for Domain Generalization: A Mixup-guided Solution

arXiv 2209.00652 ↗First posted 1 Sep 2022cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

The distribution shifts between training and test data typically undermine the performance of models. In recent years, lots of work pays attention to domain generalization (DG) where distribution shifts exist, and target data are unseen. Despite the progress in algorithm design, two foundational factors have long been ignored: 1) the optimization for regularization-based objectives, and 2) the model selection for DG since no knowledge about the target domain can be utilized. In this paper, we propose Mixup guided optimization and selection techniques for DG. For optimization, we utilize an adapted Mixup to generate an out-of-distribution dataset that can guide the preference direction and optimize with Pareto optimization. For model selection, we generate a validation dataset with a closer distance to the target distribution, and thereby it can better represent the target data. We also present some theoretical insights behind our proposals. Comprehensive experiments demonstrate that our model optimization and selection techniques can largely improve the performance of existing domain generalization algorithms and even achieve new state-of-the-art results.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 34 of its 59 references that are in the map. First posted on arXiv 1 Sep 2022.

Builds on

34 of 59 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  4. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,906
  5. 2019Invariant Risk Minimizationcited by 1,832
  6. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
  7. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
  8. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,194
  9. 2020In Search of Lost Domain Generalizationcited by 1,042
  10. 2017Deeper, Broader and Artier Domain Generalizationcited by 983
  11. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 816
  12. 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 762
22 more
  1. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 762
  2. 2021Domain Generalization with MixStylecited by 563
  3. 2020Self-Challenging Improves Cross-Domain Generalizationcited by 426
  4. 2018A Survey of Unsupervised Deep Domain Adaptationcited by 361
  5. 2021A Fourier-based Framework for Domain Generalizationcited by 287
  6. 2018Generalizing Across Domains via Cross-Gradient Trainingcited by 268
  7. 2021Gradient Matching for Domain Generalizationcited by 258
  8. 2020Domain Generalization using Causal Matchingcited by 224
  9. 2022Improving Out-of-Distribution Robustness via Selective Augmentationcited by 202
  10. 2021Fishr: Invariant Gradient Variances for Out-of-Distribution Generalizationcited by 186
  11. 2020Single-Side Domain Generalization for Face Anti-Spoofingcited by 139
  12. 2020Learning explanations that are hard to varycited by 132
  13. 2018Solving Linear Programs in the Current Matrix Multiplication Timecited by 100
  14. 2021Towards a Theoretical Framework of Out-of-Distribution Generalizationcited by 94
  15. 2021Domain Adversarial Neural Networks for Domain Generalization: When It Works and How to Improvecited by 46
  16. 2020Auxiliary Learning by Implicit Differentiationcited by 45
  17. 2017Stratified Transfer Learning for Cross-domain Activity Recognitioncited by 40
  18. 2021Domain Generalization through Audio-Visual Relative Norm Alignment in First Person Action Recognitioncited by 35
  19. 2021Pareto Domain Adaptationcited by 20
  20. 2021Out-of-Domain Generalization from a Single Source: An Uncertainty Quantification Approachcited by 18
  21. 2020An Early Study on Intelligent Analysis of Speech under COVID-19: Severity, Sleep Quality, Fatigue, and Anxietycited by 11
  22. 2022On the Limitations of General Purpose Domain Generalisation Methodscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 1; 2025: 1; 2026: 2.

6 papers.

  1. 2023Learning Diverse Features in Vision Transformers for Improved Generalizationcited by 4
  2. 2025HAROOD: A Benchmark for Out-of-distribution Generalization in Sensor-based Human Activity Recognitioncited by 3
  3. 2023SSL Framework for Causal Inconsistency between Structures and Representationscited by 2
  4. 2024Clustering-Based Validation Splits for Model Selection under Domain Shiftcited by 0
  5. 2026Negatives-Dominant Contrastive Learning for Generalization in Imbalanced Domainscited by 0
  6. 2026Reliability-Aware Checkpoint Selection for Domain Generalizationcited by 0