mlsanityData to 7 Oct 2026

Domain Generalization via Nuclear Norm Regularization

arXiv 2303.07527 ↗First posted 13 Mar 2023cs.LG cs.CVcited by 12

Not on the map. Why?

Abstract (arXiv)

The ability to generalize to unseen domains is crucial for machine learning systems deployed in the real world, especially when we only have data from limited training domains. In this paper, we propose a simple and effective regularization method based on the nuclear norm of the learned features for domain generalization. Intuitively, the proposed regularizer mitigates the impacts of environmental features and encourages learning domain-invariant features. Theoretically, we provide insights into why nuclear norm regularization is more effective compared to ERM and alternative regularization methods. Empirically, we conduct extensive experiments on both synthetic and real datasets. We show nuclear norm regularization achieves strong performance compared to baselines in a wide range of domain generalization tasks. Moreover, our regularizer is broadly applicable with various methods such as ERM and SWAD with consistently improved performance, e.g., 1.7% and 0.9% test accuracy improvements respectively on the DomainBed benchmark.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 49 of its 77 references that are in the map. First posted on arXiv 13 Mar 2023.

Builds on

49 of 77 references.

References · 49

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  3. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  4. 2019Invariant Risk Minimizationcited by 1,832
  5. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  6. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  7. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  8. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,327
  9. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
  10. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,194
  11. 2017Deep Hashing Network for Unsupervised Domain Adaptationcited by 1,181
  12. 2020In Search of Lost Domain Generalizationcited by 1,042
37 more
  1. 2017Deeper, Broader and Artier Domain Generalizationcited by 983
  2. 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 762
  3. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 737
  4. 2018Recognition in Terra Incognitacited by 655
  5. 2021Domain Generalization with MixStylecited by 563
  6. 2021Just Train Twice: Improving Group Robustness without Training Group Informationcited by 522
  7. 2021SWAD: Domain Generalization by Seeking Flat Minimacited by 431
  8. 2020Self-Challenging Improves Cross-Domain Generalizationcited by 426
  9. 2019Domain Generalization via Model-Agnostic Learning of Semantic Featurescited by 405
  10. 2022Last Layer Re-Training is Sufficient for Robustness to Spurious Correlationscited by 379
  11. 2018Confounding variables can degrade generalization performance of radiological deep learning modelscited by 374
  12. 2020Noise or Signal: The Role of Image Backgrounds in Object Recognitioncited by 342
  13. 2020Learning to Generate Novel Domains for Domain Generalizationcited by 296
  14. 2020The Risks of Invariant Risk Minimizationcited by 262
  15. 2021Gradient Matching for Domain Generalizationcited by 258
  16. 2022Improving Out-of-Distribution Robustness via Selective Augmentationcited by 202
  17. 2019Learning De-biased Representations with Biased Representationscited by 199
  18. 2021SelfReg: Self-supervised Contrastive Regularization for Domain Generalizationcited by 189
  19. 2021Fishr: Invariant Gradient Variances for Out-of-Distribution Generalizationcited by 186
  20. 2020Improve Unsupervised Domain Adaptation with Mixup Trainingcited by 185
  21. 2017Domain Generalization by Marginal Transfer Learningcited by 180
  22. 2022Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlationscited by 171
  23. 2021Ensemble of Averages: Improving Model Selection and Boosting Performance in Domain Generalizationcited by 144
  24. 2022Domain Generalization by Mutual-Information Regularization with Pre-trained Modelscited by 133
  25. 2020Learning explanations that are hard to varycited by 132
  26. 2018Tensor Robust Principal Component Analysis with A New Tensor Nuclear Normcited by 110
  27. 2022Is Out-of-Distribution Detection Learnable?cited by 109
  28. 2020Heterogeneous Domain Generalization via Domain Mixupcited by 103
  29. 2021Exploiting Domain-Specific Features to Enhance Domain Generalizationcited by 86
  30. 2022Sparse Mixture-of-Experts are Domain Generalizable Learnerscited by 85
  31. 2022Domain-Adjusted Regression or: ERM May Already Learn Features Sufficient for Out-of-Distribution Generalizationcited by 69
  32. 2021On the Impact of Spurious Correlation for Out-of-distribution Detectioncited by 68
  33. 2021Towards Unsupervised Domain Generalizationcited by 40
  34. 2016Object Recognition with and without Objectscited by 39
  35. 2022Attention Diversification for Domain Generalizationcited by 29
  36. 2023The Trade-off between Universality and Label Efficiency of Representations from Contrastive Learningcited by 27
  37. 2018Convex Relaxation Methods for Community Detectioncited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 3; 2026: 2.

12 papers.

  1. 2024Why Larger Language Models Do In-context Learning Differently?cited by 44
  2. 2024Do Large Language Models Have Compositional Ability? An Investigation into Limitations and Scalabilitycited by 39
  3. 2024Towards Few-Shot Adaptation of Foundation Models via Multitask Finetuningcited by 31
  4. 2024Unraveling the Smoothness Properties of Diffusion Models: A Gaussian Mixture Perspectivecited by 24
  5. 2024Differential Privacy Mechanisms in Neural Tangent Kernel Regressioncited by 14
  6. 2025Theoretical Guarantees for High Order Trajectory Refinement in Generative Flowscited by 9
  7. 2025When Can We Solve the Weighted Low Rank Approximation Problem in Truly Subquadratic Time?cited by 3
  8. 2024Bridging Domains with Approximately Shared Featurescited by 2
  9. 2024Efficient Unsupervised Domain Adaptation Regression for Spatial-Temporal Sensor Fusioncited by 2
  10. 2026SLORR: Simple and Efficient In-Training Low-Rank Regularizationcited by 1
  11. 2025Structured Universal Adversarial Attacks on Object Detection for Video Sequencescited by 0
  12. 2026Fast Differentiable SVD on GPU via Polar Decompositioncited by 0