mlsanityData to 7 Oct 2026

Robust Generalization against Photon-Limited Corruptions via Worst-Case Sharpness Minimization

arXiv 2303.13087 ↗First posted 23 Mar 2023cs.CVcited by 29

Not on the map. Why?

Abstract (arXiv)

Robust generalization aims to tackle the most challenging data distributions which are rare in the training set and contain severe noises, i.e., photon-limited corruptions. Common solutions such as distributionally robust optimization (DRO) focus on the worst-case empirical risk to ensure low training error on the uncommon noisy distributions. However, due to the over-parameterized model being optimized on scarce worst-case data, DRO fails to produce a smooth loss landscape, thus struggling on generalizing well to the test set. Therefore, instead of focusing on the worst-case risk minimization, we propose SharpDRO by penalizing the sharpness of the worst-case distribution, which measures the loss changes around the neighbor of learning parameters. Through worst-case sharpness minimization, the proposed method successfully produces a flat loss curve on the corrupted distributions, thus achieving robust generalization. Moreover, by considering whether the distribution annotation is available, we apply SharpDRO to two problem settings and design a worst-case selection process for robust generalization. Theoretically, we show that SharpDRO has a great convergence guarantee. Experimentally, we simulate photon-limited corruptions using CIFAR10/100 and ImageNet30 datasets and show that SharpDRO exhibits a strong generalization ability against severe corruptions and exceeds well-known baseline methods with large performance gains.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 56 of its 72 references that are in the map. First posted on arXiv 23 Mar 2023.

Builds on

56 of 72 references.

References · 56

  1. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2016Wide Residual Networkscited by 3,057
  4. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  5. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  6. 2019Invariant Risk Minimizationcited by 1,832
  7. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  8. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
  9. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,327
  10. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  11. 2015Learning Transferable Features with Deep Adaptation Networkscited by 1,323
  12. 2017Adversarial Discriminative Domain Adaptationcited by 1,309
38 more
  1. 2020Energy-based Out-of-distribution Detectioncited by 1,296
  2. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
  3. 2018Deep Anomaly Detection with Outlier Exposurecited by 939
  4. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 737
  5. 2013Domain Generalization via Invariant Feature Representationcited by 622
  6. 2021Just Train Twice: Improving Group Robustness without Training Group Informationcited by 522
  7. 2019Domain Generalization by Solving Jigsaw Puzzlescited by 449
  8. 2021SWAD: Domain Generalization by Seeking Flat Minimacited by 431
  9. 2018Fairness Without Demographics in Repeated Loss Minimizationcited by 363
  10. 2016Entropy-SGD: Biasing Gradient Descent Into Wide Valleyscited by 357
  11. 2018Learning Models with Uniform Performance via Distributionally Robust Optimizationcited by 334
  12. 2020Environment Inference for Invariant Learningcited by 306
  13. 2020Learning to Learn Single Domain Generalizationcited by 295
  14. 2022Mitigating Neural Network Overconfidence with Logit Normalizationcited by 251
  15. 2020Domain Generalization using Causal Matchingcited by 224
  16. 2019Are Anchor Points Really Indispensable in Label-Noise Learning?cited by 208
  17. 2021Towards Causal Representation Learningcited by 207
  18. 2019Solving a Class of Non-Convex Min-Max Games Using Iterative First Order Methodscited by 148
  19. 2021Efficient Sharpness-aware Minimization for Improved Training of Neural Networkscited by 137
  20. 2020Invariant Rationalizationcited by 131
  21. 2022Towards Efficient and Scalable Sharpness-Aware Minimizationcited by 129
  22. 2022Penalizing Gradient Norm for Efficiently Improving Generalization in Deep Learningcited by 122
  23. 2021Open Domain Generalization with Domain-Augmented Meta-Learningcited by 115
  24. 2022Sharpness-Aware Training for Freecited by 95
  25. 2022Fisher SAM: Information Geometry and Sharpness Aware Minimisationcited by 84
  26. 2020Regularizing Neural Networks via Adversarial Model Perturbationcited by 78
  27. 2022Make Sharpness-Aware Minimization Stronger: A Sparsified Perturbation Approachcited by 70
  28. 2021Probabilistic Margins for Instance Reweighting in Adversarial Trainingcited by 57
  29. 2021DORO: Distributional and Outlier Robust Optimizationcited by 54
  30. 2023FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracycited by 49
  31. 2021Faster Single-loop Algorithms for Minimax Optimization without Strong Concavitycited by 47
  32. 2022Harnessing Out-Of-Distribution Examples via Augmenting Content and Stylecited by 43
  33. 2022Improving Sharpness-Aware Minimization with Fisher Mask for Better Generalization on Language Modelscited by 42
  34. 2023Out-of-distribution Detection with Implicit Outlier Transformationcited by 42
  35. 2020Part-dependent Label Noise: Towards Instance-dependent Label Noisecited by 40
  36. 2021Towards Unsupervised Domain Generalizationcited by 40
  37. 2021Transporting Causal Mechanisms for Unsupervised Domain Adaptationcited by 38
  38. 2022Improving Task-free Continual Learning by Distributionally Robust Memory Evolutioncited by 29

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 9; 2025: 8; 2026: 4.

29 papers.

Sort:

By area:

  1. 2025Exploring Criteria of Loss Reweighting to Enhance LLM Unlearningcited by 39
  2. 2023Learning to Augment Distributions for Out-of-Distribution Detectioncited by 38
  3. 2023Out-of-distribution Detection Learning with Unreliable Out-of-distribution Sourcescited by 31
  4. 2023IDEAL: Influence-Driven Selective Annotations Empower In-Context Learners in Large Language Modelscited by 26
  5. 2023FlatMatch: Bridging Labeled Data and Unlabeled Data with Cross-Sharpness for Semi-Supervised Learningcited by 20
  6. 2024HYPO: Hyperspherical Out-of-Distribution Generalizationcited by 13
  7. 2023Winning Prize Comes from Losing Tickets: Improve Invariant Learning by Exploring Variant Parameters for Out-of-Distribution Generalizationcited by 12
  8. 2024Eliminating Catastrophic Overfitting Via Abnormal Adversarial Examples Regularizationcited by 11
  9. 2025GRU: Mitigating the Trade-off between Unlearning and Retention for LLMscited by 11
  10. 2024Layer-Aware Analysis of Catastrophic Overfitting: Revealing the Pseudo-Robust Shortcut Dependencycited by 9
  11. 2023Stability and Generalization of the Decentralized Stochastic Gradient Descent Ascent Algorithmcited by 8
  12. 2025Jailbreaking the Non-Transferable Barrier via Test-Time Data Disguisingcited by 8
17 more
  1. 2024AHA: Human-Assisted Out-of-Distribution Generalization and Detectioncited by 7
  2. 2025Toward Robust Non-Transferable Learning: A Survey and Benchmarkcited by 7
  3. 2024Mitigating Label Noise on Graph via Topological Sample Selectioncited by 6
  4. 2024Out-of-Distribution Learning with Human Feedbackcited by 6
  5. 2024From Optimization to Generalization: Fair Federated Learning against Quality Shift via Inter-Client Sharpness Matchingcited by 5
  6. 2025When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Needcited by 5
  7. 2026Is Gradient Ascent Really Necessary? Memorize to Forget for Machine Unlearningcited by 5
  8. 2023OODRobustBench: a Benchmark and Large-Scale Analysis of Adversarial Robustness under Distribution Shiftcited by 4
  9. 2023Critical Influence of Overparameterization on Sharpness-aware Minimizationcited by 3
  10. 2024Bridging OOD Detection and Generalization: A Graph-Theoretic Viewcited by 3
  11. 2026i-PhysGaussian: Implicit Physical Simulation for 3D Gaussian Splattingcited by 3
  12. 2025FedPCA: Noise-Robust Fair Federated Learning via Performance-Capacity Analysiscited by 2
  13. 2026Intellectual Property Protection for 3D Gaussian Splatting Assets: A Surveycited by 2
  14. 2025Distributionally Robust Optimization with Adversarial Data Contaminationcited by 1
  15. 2024Making Robust Generalizers Less Rigid with Loss Concentrationcited by 0
  16. 2025Fair Federated Medical Image Classification Against Quality Shift via Inter-Client Progressive State Matchingcited by 0
  17. 2026BrokenBind: Universal Modality Exploration beyond Dataset Boundariescited by 0