mlsanityData to 7 Oct 2026

When Neural Networks Fail to Generalize? A Model Sensitivity Perspective

arXiv 2212.00850 ↗First posted 1 Dec 2022cs.CV cs.AIcited by 10

Not on the map. Why?

Abstract (arXiv)

Domain generalization (DG) aims to train a model to perform well in unseen domains under different distributions. This paper considers a more realistic yet more challenging scenario,namely Single Domain Generalization (Single-DG), where only a single source domain is available for training. To tackle this challenge, we first try to understand when neural networks fail to generalize? We empirically ascertain a property of a model that correlates strongly with its generalization that we coin as "model sensitivity". Based on our analysis, we propose a novel strategy of Spectral Adversarial Data Augmentation (SADA) to generate augmented images targeted at the highly sensitive frequencies. Models trained with these hard-to-learn samples can effectively suppress the sensitivity in the frequency space, which leads to improved generalization performance. Extensive experiments on multiple public datasets demonstrate the superiority of our approach, which surpasses the state-of-the-art single-DG methods.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 36 of its 51 references that are in the map. First posted on arXiv 1 Dec 2022.

Builds on

36 of 51 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  3. 2016Wide Residual Networkscited by 3,057
  4. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  5. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  6. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  7. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,906
  8. 2019Invariant Risk Minimizationcited by 1,832
  9. 2017Deeper, Broader and Artier Domain Generalizationcited by 983
  10. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 897
  11. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 762
  12. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 737
24 more
  1. 2013Domain Generalization via Invariant Feature Representationcited by 622
  2. 2020FDA: Fourier Domain Adaptation for Semantic Segmentationcited by 540
  3. 2018Generalizing to Unseen Domains via Adversarial Data Augmentationcited by 451
  4. 2019Domain Generalization by Solving Jigsaw Puzzlescited by 449
  5. 2019Domain Generalization via Model-Agnostic Learning of Semantic Featurescited by 405
  6. 2017Unified Deep Supervised Domain Adaptation and Generalizationcited by 322
  7. 2020Learning to Learn Single Domain Generalizationcited by 295
  8. 2019A Fourier Perspective on Model Robustness in Computer Visioncited by 288
  9. 2021A Fourier-based Framework for Domain Generalizationcited by 287
  10. 2021FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Spacecited by 284
  11. 2020The Risks of Invariant Risk Minimizationcited by 262
  12. 2021Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalizationcited by 220
  13. 2021Learning to Diversify for Single Domain Generalizationcited by 170
  14. 2020Robust and Generalizable Visual Representation Learning via Random Convolutionscited by 141
  15. 2021PixMix: Dreamlike Pictures Comprehensively Improve Safety Measurescited by 131
  16. 2021Fair Mixup: Fairness via Interpolationcited by 109
  17. 2021AugMax: Adversarial Composition of Random Augmentations for Robust Trainingcited by 89
  18. 2020Empirical or Invariant Risk Minimization? A Sample Complexity Perspectivecited by 62
  19. 2021OTCE: A Transferability Metric for Cross-Domain Cross-Task Representationscited by 62
  20. 2020When is invariance useful in an Out-of-Distribution Generalization problem ?cited by 52
  21. 2019d-SNE: Domain Adaptation using Stochastic Neighborhood Embeddingcited by 40
  22. 2021Certified Adversarial Defenses Meet Out-of-Distribution Corruptions: Benchmarking Robustness and Simple Baselinescited by 17
  23. 2021Task-Oriented Low-Dose CT Image Denoisingcited by 11
  24. 2020Robustified Domain Adaptationcited by 2

Built on by

Cited by year

Citing papers per year: 2023: 5; 2024: 4; 2025: 1.

10 papers.

  1. 2024LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuningcited by 133
  2. 2023High Fidelity 3D Hand Shape Reconstruction via Scalable Graph Frequency Decompositioncited by 18
  3. 2024Cardiovascular Disease Detection from Multi-View Chest X-rays with BI-Mambacited by 9
  4. 2023Spectral Adversarial MixUp for Few-Shot Unsupervised Domain Adaptationcited by 8
  5. 2023Web Content Filtering through knowledge distillation of Large Language Modelscited by 7
  6. 2024Disease-informed Adaptation of Vision-Language Modelscited by 7
  7. 2024Towards Combating Frequency Simplicity-biased Learning for Domain Generalizationcited by 6
  8. 2023Can We Reliably Improve the Robustness to Image Acquisition of Remote Sensing of PV Systems?cited by 2
  9. 2025SELF: Self-Extend the Context Length With Logistic Growth Functioncited by 1
  10. 2023Soft-tissue Driven Craniomaxillofacial Surgical Planningcited by 0