mlsanityData to 7 Oct 2026

Generalizing to Unseen Domains: A Survey on Domain Generalization

arXiv 2103.03097 ↗First posted 2 Mar 2021cs.LG cs.AI cs.CVcited by 762

Not on the map. Why?

Abstract (arXiv)

Machine learning systems generally assume that the training and testing distributions are the same. To this end, a key requirement is to develop models that can generalize to unseen distributions. Domain generalization (DG), i.e., out-of-distribution generalization, has attracted increasing interests in recent years. Domain generalization deals with a challenging setting where one or several different but related domain(s) are given, and the goal is to learn a model that can generalize to an unseen test domain. Great progress has been made in the area of domain generalization for years. This paper presents the first review of recent advances in this area. First, we provide a formal definition of domain generalization and discuss several related fields. We then thoroughly review the theories related to domain generalization and carefully analyze the theory behind generalization. We categorize recent algorithms into three classes: data manipulation, representation learning, and learning strategy, and present several popular algorithms in detail for each category. Third, we introduce the commonly used datasets, applications, and our open-sourced codebase for fair evaluation. Finally, we summarize existing literature and present some potential research topics for the future.

Key facts. Cited by 762 papers in the map (data through 7 Oct 2026). Builds on 134 of its 290 references that are in the map. First posted on arXiv 2 Mar 2021.

Builds on

134 of 290 references.

References · 134

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  5. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  6. 2013Auto-Encoding Variational Bayescited by 3,980
  7. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  8. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  9. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  10. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  11. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
  12. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
38 more
  1. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,906
  2. 2019Invariant Risk Minimizationcited by 1,832
  3. 2016OpenAI Gymcited by 1,684
  4. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  5. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  6. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,327
  7. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
  8. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,194
  9. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,187
  10. 2017Deep Hashing Network for Unsupervised Domain Adaptationcited by 1,181
  11. 2019A Comprehensive Survey on Transfer Learningcited by 1,154
  12. 2020In Search of Lost Domain Generalizationcited by 1,042
  13. 2017Deeper, Broader and Artier Domain Generalizationcited by 983
  14. 2020Meta-Learning in Neural Networks: A Surveycited by 944
  15. 2016Playing for Data: Ground Truth from Computer Gamescited by 816
  16. 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 762
  17. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 737
  18. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
  19. 2019Self-supervised Visual Feature Learning with Deep Neural Networks: A Surveycited by 656
  20. 2018Recognition in Terra Incognitacited by 655
  21. 2018Deep Visual Domain Adaptation: A Surveycited by 634
  22. 2013Domain Generalization via Invariant Feature Representationcited by 622
  23. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  24. 2014Deep Domain Confusion: Maximizing for Domain Invariancecited by 597
  25. 2018Generalizing to Unseen Domains via Adversarial Data Augmentationcited by 451
  26. 2019Domain Generalization by Solving Jigsaw Puzzlescited by 449
  27. 2021SWAD: Domain Generalization by Seeking Flat Minimacited by 431
  28. 2020Self-Challenging Improves Cross-Domain Generalizationcited by 426
  29. 2019Domain Generalization via Model-Agnostic Learning of Semantic Featurescited by 405
  30. 2019Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classificationcited by 396
  31. 2015Return of Frustratingly Easy Domain Adaptationcited by 389
  32. 2017Wasserstein Auto-Encoderscited by 371
  33. 2018Two at Once: Enhancing Learning and Generalization Capacities via IBN-Netcited by 361
  34. 2016A Learned Representation For Artistic Stylecited by 340
  35. 2017Functional Map of the Worldcited by 338
  36. 2017Unified Deep Supervised Domain Adaptation and Generalizationcited by 322
  37. 2020Learning to Generate Novel Domains for Domain Generalizationcited by 296
  38. 2020Learning to Learn Single Domain Generalizationcited by 295

50 most cited of 134 All 134 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 34; 2022: 144; 2023: 200; 2024: 157; 2025: 150; 2026: 77.

762 papers.

Sort:

By area:

  1. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  2. 2022Teaching Models to Express Their Uncertainty in Wordscited by 731
  3. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 431
  4. 2023A Comprehensive Survey on Test-Time Adaptation under Distribution Shiftscited by 353
  5. 2023On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspectivecited by 208
  6. 2021Trustworthy AI: From Principles to Practicescited by 206
  7. 2023Robust Test-Time Adaptation in Dynamic Scenarioscited by 185
  8. 2022Parameter-free Online Test-time Adaptationcited by 184
  9. 2023ZipIt! Merging Models from Different Tasks without Trainingcited by 178
  10. 2022Learning Causally Invariant Representations for Out-of-Distribution Generalization on Graphscited by 144
  11. 2022GAPartNet: Cross-Category Domain-Generalizable Object Perception and Manipulation via Generalizable and Actionable Partscited by 135
  12. 2022Uncertainty Modeling for Out-of-Distribution Generalizationcited by 122
38 more
  1. 2022Exact Feature Distribution Matching for Arbitrary Style Transfer and Domain Generalizationcited by 122
  2. 2021AdaRNN: Adaptive Learning and Forecasting of Time Seriescited by 120
  3. 2023Diffusion Models for Image Restoration and Enhancement: A Comprehensive Surveycited by 108
  4. 2022Out-Of-Distribution Generalization on Graphs: A Surveycited by 97
  5. 2023Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmarkcited by 97
  6. 2022USB: A Unified Semi-supervised Learning Benchmark for Classificationcited by 95
  7. 2021Towards a Theoretical Framework of Out-of-Distribution Generalizationcited by 94
  8. 2021OoD-Bench: Quantifying and Understanding Two Dimensions of Out-of-Distribution Generalizationcited by 86
  9. 2022NICO++: Towards Better Benchmarking for Domain Generalizationcited by 86
  10. 2022Self-Supervised Learning for Videos: A Surveycited by 85
  11. 2023Towards Large-scale 3D Representation Learning with Multi-dataset Point Prompt Trainingcited by 85
  12. 2022Shortcut Learning of Large Language Models in Natural Language Understandingcited by 83
  13. 2024Robust agents learn causal world modelscited by 81
  14. 2024LLM Internal States Reveal Hallucination Risk Faced With a Querycited by 80
  15. 2023Federated Learning for Medical Image Analysis: A Surveycited by 77
  16. 2022GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-distribution Generalization Perspectivecited by 72
  17. 2023FedCLIP: Fast Generalization and Personalization for CLIP in Federated Learningcited by 70
  18. 2025STAR-1: Safer Alignment of Reasoning LLMs with 1K Datacited by 67
  19. 2022Test-Time Adaptation via Self-Training with Nearest Neighbor Informationcited by 64
  20. 2023FreeMask: Synthetic Images with Dense Annotations Make Stronger Segmentation Modelscited by 61
  21. 2023Generalized Video Anomaly Event Detection: Systematic Taxonomy and Comparison of Deep Modelscited by 59
  22. 2023PromptStyler: Prompt-driven Style Generation for Source-free Domain Generalizationcited by 59
  23. 2023Feed Two Birds with One Scone: Exploiting Wild Data for Both Out-of-Distribution Generalization and Detectioncited by 57
  24. 2023Label-Free Liver Tumor Segmentationcited by 56
  25. 2022ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalizationcited by 54
  26. 2025SMART: Self-Aware Agent for Tool Overuse Mitigationcited by 52
  27. 2022Generalizing to New Physical Systems via Context-Informed Dynamics Modelcited by 49
  28. 2022Evaluation Gaps in Machine Learning Practicecited by 48
  29. 2023SimMMDG: A Simple and Effective Framework for Multi-modal Domain Generalizationcited by 48
  30. 2023Rethinking Domain Generalization for Face Anti-spoofing: Separability and Alignmentcited by 47
  31. 2023Domain Generalization for Medical Image Analysis: A Reviewcited by 47
  32. 2021Learning Disentangled Representations in the Imaging Domaincited by 46
  33. 2022Diagnosing failures of fairness transfer across distribution shift in real-world medical settingscited by 45
  34. 2022Out-of-distribution Generalization with Causal Invariant Transformationscited by 45
  35. 20213D-VField: Adversarial Augmentation of Point Clouds for Domain Generalization in 3D Object Detectioncited by 43
  36. 2023BayeSeg: Bayesian Modeling for Medical Image Segmentation with Interpretable Generalizabilitycited by 42
  37. 2021Federated Learning with Domain Generalizationcited by 41
  38. 2022Compound Domain Generalization via Meta-Knowledge Encodingcited by 40

50 most cited of 762 All 762 (JSON)