mlsanityData to 7 Oct 2026

Model-Based Domain Generalization

arXiv 2102.11436 ↗First posted 23 Feb 2021stat.ML cs.AI cs.LGcited by 103

Not on the map. Why?

Abstract (arXiv)

Despite remarkable success in a variety of applications, it is well-known that deep learning can fail catastrophically when presented with out-of-distribution data. Toward addressing this challenge, we consider the domain generalization problem, wherein predictors are trained using data drawn from a family of related training domains and then evaluated on a distinct and unseen test domain. We show that under a natural model of data generation and a concomitant invariance condition, the domain generalization problem is equivalent to an infinite-dimensional constrained statistical learning problem; this problem forms the basis of our approach, which we call Model-Based Domain Generalization. Due to the inherent challenges in solving constrained optimization problems in deep learning, we exploit nonconvex duality theory to develop unconstrained relaxations of this statistical problem with tight bounds on the duality gap. Based on this theoretical motivation, we propose a novel domain generalization algorithm with convergence guarantees. In our experiments, we report improvements of up to 30 percentage points over state-of-the-art domain generalization baselines on several benchmarks including ColoredMNIST, Camelyon17-WILDS, FMoW-WILDS, and PACS.

Key facts. Cited by 103 papers in the map (data through 7 Oct 2026). Builds on 114 of its 181 references that are in the map. First posted on arXiv 23 Feb 2021.

Builds on

114 of 181 references.

References · 114

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2016Densely Connected Convolutional Networkscited by 6,691
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  6. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  7. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  8. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  9. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  10. 2013Auto-Encoding Variational Bayescited by 3,980
  11. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  12. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
38 more
  1. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  2. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  3. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  4. 2019Invariant Risk Minimizationcited by 1,832
  5. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  6. 2015Spatial Transformer Networkscited by 1,423
  7. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  8. 2017Adversarial Discriminative Domain Adaptationcited by 1,309
  9. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
  10. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,194
  11. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,095
  12. 2020In Search of Lost Domain Generalizationcited by 1,042
  13. 2017Deeper, Broader and Artier Domain Generalizationcited by 983
  14. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 897
  15. 2018Multimodal Unsupervised Image-to-Image Translationcited by 795
  16. 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 762
  17. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 737
  18. 2017Evasion Attacks against Machine Learning at Test Timecited by 686
  19. 2018Deep Visual Domain Adaptation: A Surveycited by 634
  20. 2013Domain Generalization via Invariant Feature Representationcited by 622
  21. 2017OptNet: Differentiable Optimization as a Layer in Neural Networkscited by 588
  22. 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 564
  23. 2020Measuring Robustness to Natural Distribution Shifts in Image Classificationcited by 452
  24. 2018Generalizing to Unseen Domains via Adversarial Data Augmentationcited by 451
  25. 2019Likelihood Ratios for Out-of-Distribution Detectioncited by 451
  26. 2019Domain Generalization by Solving Jigsaw Puzzlescited by 449
  27. 2020Self-Challenging Improves Cross-Domain Generalizationcited by 426
  28. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 424
  29. 2017Wild Patterns: Ten Years After the Rise of Adversarial Machine Learningcited by 411
  30. 2019Domain Generalization via Model-Agnostic Learning of Semantic Featurescited by 405
  31. 2018Detecting and Correcting for Label Shift with Black Box Predictorscited by 364
  32. 2020Noise or Signal: The Role of Image Backgrounds in Object Recognitioncited by 342
  33. 2017Unified Deep Supervised Domain Adaptation and Generalizationcited by 322
  34. 2020Learning to Learn Single Domain Generalizationcited by 295
  35. 2018Towards a Definition of Disentangled Representationscited by 294
  36. 2018Generalizing Across Domains via Cross-Gradient Trainingcited by 268
  37. 2020The Risks of Invariant Risk Minimizationcited by 262
  38. 2015Domain Generalization for Object Recognition with Multi-task Autoencoderscited by 253

50 most cited of 114 All 114 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 25; 2023: 29; 2024: 24; 2025: 8; 2026: 3.

103 papers.

Sort:

By area:

  1. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 762
  2. 2021Domain Generalization: A Surveycited by 760
  3. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  4. 2021Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalizationcited by 248
  5. 2021Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalizationcited by 220
  6. 2022Domain Generalization by Mutual-Information Regularization with Pre-trained Modelscited by 133
  7. 2021Towards Principled Disentanglement for Domain Generalizationcited by 91
  8. 2021AugMax: Adversarial Composition of Random Augmentations for Robust Trainingcited by 89
  9. 2021OoD-Bench: Quantifying and Understanding Two Dimensions of Out-of-Distribution Generalizationcited by 86
  10. 2021Extending the WILDS Benchmark for Unsupervised Adaptationcited by 85
  11. 2021Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Testscited by 77
  12. 2022Probable Domain Generalization via Quantile Risk Minimizationcited by 66
38 more
  1. 2022ID and OOD Performance Are Sometimes Inversely Correlated on Real-world Datasetscited by 44
  2. 2023Leveraging Vision-Language Models for Improving Domain Generalization in Image Classificationcited by 41
  3. 2021Adversarial Robustness with Semi-Infinite Constrained Learningcited by 39
  4. 2022Probabilistically Robust Learning: Balancing Average- and Worst-case Performancecited by 36
  5. 2023Subspace Identification for Multi-Source Domain Adaptationcited by 36
  6. 2021Universal Approximation Under Constraints is Possible with Transformerscited by 32
  7. 2021Iterative Feature Matching: Toward Provable Domain Generalization with Logarithmic Environmentscited by 30
  8. 2022WOODS: Benchmarks for Out-of-Distribution Generalization in Time Seriescited by 29
  9. 2023Out-of-Domain Robustness via Targeted Augmentationscited by 24
  10. 2022How to Fine-Tune Vision Models with SGDcited by 23
  11. 2021Understanding the Generalization Benefit of Model Invariance from a Data Perspectivecited by 22
  12. 2023MADG: Margin-based Adversarial Learning for Domain Generalizationcited by 22
  13. 2022Explicit Tradeoffs between Adversarial and Natural Distributional Robustnesscited by 20
  14. 2022Automatic Data Augmentation via Invariance-Constrained Learningcited by 17
  15. 2023Generalizable Sleep Staging via Multi-Level Domain Alignmentcited by 17
  16. 2022Toward Certified Robustness Against Real-World Distribution Shiftscited by 16
  17. 2022Improving Diversity with Adversarially Learned Transformations for Domain Generalizationcited by 16
  18. 2023Resilient Constrained Learningcited by 15
  19. 2022FL Games: A Federated Learning Framework for Distribution Shiftscited by 14
  20. 2023Towards Artificial General Intelligence (AGI) in the Internet of Things (IoT): Opportunities and Challengescited by 14
  21. 2023Data Augmentations for Improved (Large) Language Model Generalizationcited by 14
  22. 2022Spuriosity Rankings: Sorting Data to Measure and Mitigate Biasescited by 13
  23. 2023Adversarial Training Should Be Cast as a Non-Zero-Sum Gamecited by 13
  24. 2022Do Deep Networks Transfer Invariances Across Classes?cited by 12
  25. 2024CDAD-Net: Bridging Domain Gaps in Generalized Category Discoverycited by 12
  26. 2022An Empirical Study on Distribution Shift Robustness From the Perspective of Pre-Training and Data Augmentationcited by 11
  27. 2023Towards Fair Disentangled Online Learning for Changing Environmentscited by 11
  28. 2024Free Lunch in Pathology Foundation Model: Task-specific Model Adaptation with Concept-Guided Feature Enhancementcited by 11
  29. 2024LLM-Based World Models Can Make Decisions Solely, But Rigorous Evaluations are Neededcited by 10
  30. 2024Seeking Consistent Flat Minima for Better Domain Generalization via Refining Loss Landscapescited by 10
  31. 2023Cross Contrasting Feature Perturbation for Domain Generalizationcited by 9
  32. 2023Distilling Out-of-Distribution Robustness from Vision-Language Foundation Modelscited by 9
  33. 2024A Survey on Domain Generalization for Medical Image Analysiscited by 9
  34. 2023Open Compound Domain Adaptation with Object Style Compensation for Semantic Segmentationcited by 8
  35. 2023Topology-aware Robust Optimization for Out-of-distribution Generalizationcited by 7
  36. 2023Algorithmic Fairness Generalization under Covariate and Dependence Shifts Simultaneouslycited by 7
  37. 2024Beyond the Federation: Topology-aware Federated Learning for Generalization to Unseen Clientscited by 7
  38. 2024Learning Fair Invariant Representations under Covariate and Correlation Shifts Simultaneouslycited by 7

50 most cited of 103 All 103 (JSON)