mlsanityData to 7 Oct 2026

Towards Robust Out-of-Distribution Generalization Bounds via Sharpness

arXiv 2403.06392 ↗First posted 11 Mar 2024cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

Generalizing to out-of-distribution (OOD) data or unseen domain, termed OOD generalization, still lacks appropriate theoretical guarantees. Canonical OOD bounds focus on different distance measurements between source and target domains but fail to consider the optimization property of the learned model. As empirically shown in recent work, the sharpness of learned minima influences OOD generalization. To bridge this gap between optimization and OOD generalization, we study the effect of sharpness on how a model tolerates data change in domain shift which is usually captured by "robustness" in generalization. In this paper, we give a rigorous connection between sharpness and robustness, which gives better OOD guarantees for robust algorithms. It also provides a theoretical backing for "flat minima leads to better OOD generalization". Overall, we propose a sharpness-based OOD generalization bound by taking robustness into consideration, resulting in a tighter bound than non-robust guarantees. Our findings are supported by the experiments on a ridge regression model, as well as the experiments on deep learning classification tasks.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 40 of its 59 references that are in the map. First posted on arXiv 11 Mar 2024.

Builds on

40 of 59 references.

References · 40

  1. 2019Invariant Risk Minimizationcited by 1,832
  2. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  3. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  4. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
  5. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,194
  6. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
  7. 2020In Search of Lost Domain Generalizationcited by 1,042
  8. 2017Deeper, Broader and Artier Domain Generalizationcited by 983
  9. 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 762
  10. 2021SWAD: Domain Generalization by Seeking Flat Minimacited by 431
  11. 2021Gradient Descent on Neural Networks Typically Occurs at the Edge of Stabilitycited by 413
  12. 2019The generalization error of random features regression: Precise asymptotics and double descent curvecited by 410
28 more
  1. 2018Learning Models with Uniform Performance via Distributionally Robust Optimizationcited by 334
  2. 2019Bridging Theory and Algorithm for Domain Adaptationcited by 332
  3. 2020An Investigation of Why Overparameterization Exacerbates Spurious Correlationscited by 310
  4. 2017Joint Distribution Optimal Transportation for Domain Adaptationcited by 280
  5. 2020Gradient Starvation: A Learning Proclivity in Neural Networkscited by 262
  6. 2021Gradient Matching for Domain Generalizationcited by 258
  7. 2021Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalizationcited by 248
  8. 2021Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalizationcited by 220
  9. 2022Improving Out-of-Distribution Robustness via Selective Augmentationcited by 202
  10. 2021Fishr: Invariant Gradient Variances for Out-of-Distribution Generalizationcited by 186
  11. 2020Improve Unsupervised Domain Adaptation with Mixup Trainingcited by 185
  12. 2021A Fine-Grained Analysis on Distribution Shiftcited by 171
  13. 2021On Calibration and Out-of-domain Generalizationcited by 134
  14. 2015Distributionally Robust Logistic Regressioncited by 124
  15. 2020A survey on domain adaptation theory: learning bounds and theoretical guaranteescited by 109
  16. 2017Recovery Guarantees for One-hidden-layer Neural Networkscited by 94
  17. 2021Towards a Theoretical Framework of Out-of-Distribution Generalizationcited by 94
  18. 2021Heterogeneous Risk Minimizationcited by 90
  19. 2020Relative Flatness and Generalizationcited by 88
  20. 2022Understanding the Generalization Benefit of Normalization Layers: Sharpness Reductioncited by 86
  21. 2020Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracycited by 70
  22. 2018A Continuous-Time View of Early Stopping for Least Squarescited by 67
  23. 2018Learning One-hidden-layer ReLU Networks via Gradient Descentcited by 50
  24. 2021Towards Unsupervised Domain Generalizationcited by 40
  25. 2015A New PAC-Bayesian Perspective on Domain Adaptationcited by 29
  26. 2021Linear unit-tests for invariance discoverycited by 27
  27. 2016Hypothesis Transfer Learning via Transformation Functionscited by 26
  28. 2022Robustness Implies Generalization via Data-Dependent Generalization Boundscited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 5; 2026: 2.

10 papers.

  1. 2024Seeking Consistent Flat Minima for Better Domain Generalization via Refining Loss Landscapescited by 10
  2. 2025Toward Robust Non-Transferable Learning: A Survey and Benchmarkcited by 7
  3. 2024Generalization and Robustness of the Tilted Empirical Riskcited by 4
  4. 2024Towards Understanding the Role of Sharpness-Aware Minimization Algorithms for Out-of-Distribution Generalizationcited by 1
  5. 2025Flatness Guided Test-Time Adaptation for Vision-Language Modelscited by 1
  6. 2025A Validation Strategy for Deep Learning Models: Evaluating and Enhancing Robustnesscited by 1
  7. 2025Should Bias be Eliminated? A General Framework to Use Bias for OOD Generalizationcited by 0
  8. 2025Volatility in Certainty (VC): A Metric for Detecting Adversarial Perturbations During Inference in Neural Network Classifierscited by 0
  9. 2026The Price of Robustness: Stable Classifiers Need Overparameterizationcited by 0
  10. 2026Parameter-Efficient Distributionally Robust Adaptation of Tabular Foundation Models under Subpopulation Shiftcited by 0