mlsanityData to 14 Oct 2026

An Empirical Framework for Domain Generalization in Clinical Settings

arXiv 2103.11163 ↗First posted 20 Mar 2021cs.LGcited by 32

Not on the map. Why?

Abstract (arXiv)

Clinical machine learning models experience significantly degraded performance in datasets not seen during training, e.g., new hospitals or populations. Recent developments in domain generalization offer a promising solution to this problem by creating models that learn invariances across environments. In this work, we benchmark the performance of eight domain generalization methods on multi-site clinical time series and medical imaging data. We introduce a framework to induce synthetic but realistic domain shifts and sampling bias to stress-test these methods over existing non-healthcare benchmarks. We find that current domain generalization methods do not consistently achieve significant gains in out-of-distribution performance over empirical risk minimization on real-world medical imaging data, in line with prior work on general imaging datasets. However, a subset of realistic induced-shift scenarios in clinical time series data do exhibit limited performance gains. We characterize these scenarios in detail, and recommend best practices for domain generalization in the clinical setting.

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). Builds on 59 of its 87 references that are in the map. First posted on arXiv 20 Mar 2021.

Builds on

59 of 87 references.

References · 59

  1. 2016Densely Connected Convolutional Networkscited by 6,720
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  3. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,367
  4. 2016Equality of Opportunity in Supervised Learningcited by 2,007
  5. 2019Invariant Risk Minimizationcited by 1,854
  6. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,850
  7. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,423
  8. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,375
  9. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,247
  10. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,200
  11. 2020In Search of Lost Domain Generalizationcited by 1,054
  12. 2017Deeper, Broader and Artier Domain Generalizationcited by 986
38 more
  1. 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 972
  2. 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 763
  3. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 742
  4. 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 597
  5. 2015Causal inference using invariant prediction: identification and confidence intervalscited by 536
  6. 2019Domain Generalization by Solving Jigsaw Puzzlescited by 450
  7. 2019Domain Generalization via Model-Agnostic Learning of Semantic Featurescited by 407
  8. 2018Confounding variables can degrade generalization performance of radiological deep learning modelscited by 379
  9. 2019Transfusion: Understanding Transfer Learning for Medical Imagingcited by 368
  10. 2019PadChest: A large chest x-ray image dataset with multi-label annotated reportscited by 309
  11. 2018Scalable and accurate deep learning for electronic health recordscited by 280
  12. 2020The Risks of Invariant Risk Minimizationcited by 262
  13. 2019Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imagingcited by 216
  14. 2019Causality matters in medical imagingcited by 193
  15. 2020Invariant Risk Minimization Gamescited by 188
  16. 2020CheXclusion: Fairness gaps in deep chest X-ray classifierscited by 153
  17. 2017Invariant Causal Prediction for Nonlinear Modelscited by 140
  18. 2020Efficient Domain Generalization via Common-Specific Low-Rank Decompositioncited by 134
  19. 2018Anchor regression: heterogeneous data meets causalitycited by 118
  20. 2021Does Invariant Risk Minimization Capture Invariance?cited by 116
  21. 2018Invariant Representations without Adversarial Trainingcited by 112
  22. 2018Unsupervised domain adaptation for medical imaging segmentation with self-ensemblingcited by 94
  23. 2018An Empirical Study of Rich Subgroup Fairness for Machine Learningcited by 81
  24. 2020On the limits of cross-domain generalization in automated X-ray predictioncited by 62
  25. 2020Empirical or Invariant Risk Minimization? A Sample Complexity Perspectivecited by 62
  26. 2020Collaborative Unsupervised Domain Adaptation for Medical Image Diagnosiscited by 60
  27. 2019Can we trust deep learning models diagnosis? The impact of domain shift in chest radiograph classificationcited by 57
  28. 2018Training Well-Generalizing Classifiers for Fairness Metrics and Other Data-Dependent Constraintscited by 53
  29. 2020Learning Invariances in Neural Networkscited by 52
  30. 2020Removing Spurious Features can Hurt Accuracy and Affect Groups Disproportionatelycited by 52
  31. 2019Average Individual Fairness: Algorithms, Generalization and Experimentscited by 46
  32. 2020An Empirical Characterization of Fair Machine Learning For Clinical Risk Predictioncited by 45
  33. 2019Feature Robustness in Non-stationary Health Records: Caveats to Deployable Model Performance in Common Clinical Machine Learning Taskscited by 39
  34. 2019The reliability of a deep learning model in clinical out-of-distribution MRI data: a multicohort studycited by 38
  35. 2019Benchmarking machine learning models on multi-centre eICU critical care datasetcited by 35
  36. 2020An Empirical Study of Invariant Risk Minimizationcited by 35
  37. 2020Fairness and Robustness in Invariant Learning: A Case Study in Toxicity Classificationcited by 32
  38. 2020Representation via Representations: Domain Generalization via Adversarially Learned Invariant Representationscited by 26

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 11; 2023: 9; 2024: 4; 2025: 3; 2026: 3.

32 papers.

Sort:

  1. 2022Diagnosing failures of fairness transfer across distribution shift in real-world medical settingscited by 45
  2. 2022Robust and Efficient Medical Imaging with Self-Supervisioncited by 40
  3. 2022WOODS: Benchmarks for Out-of-Distribution Generalization in Time Seriescited by 30
  4. 2023ManyDG: Many-domain Generalization for Healthcare Applicationscited by 23
  5. 2022A Review of Causality for Learning Algorithms in Medical Image Analysiscited by 18
  6. 2022"Why did the Model Fail?": Attributing Model Performance Changes to Distribution Shiftscited by 18
  7. 2023When More is Less: Incorporating Additional Datasets Can Hurt Performance By Introducing Spurious Correlationscited by 18
  8. 2022Evaluating Latent Space Robustness and Uncertainty of EEG-ML Models under Realistic Distribution Shiftscited by 16
  9. 2021A comparison of approaches to improve worst-case predictive model performance over patient subpopulationscited by 12
  10. 2022Domain generalization in deep learning-based mass detection in mammography: A large-scale multi-center studycited by 10
  11. 2024A Systematic Review of Generalization Research in Medical Image Classificationcited by 8
  12. 2022DSLOB: A Synthetic Limit Order Book Dataset for Benchmarking Forecasting Algorithms under Distributional Shiftcited by 6
20 more
  1. 2023The Limits of Fair Medical Imaging AI In The Wildcited by 4
  2. 2022PhysioMTL: Personalizing Physiological Patterns using Optimal Transport Multi-Task Regressioncited by 3
  3. 2022Looking for Out-of-Distribution Environments in Multi-center Critical Care Datacited by 3
  4. 2023Cross-Shaped Windows Transformer with Self-supervised Pretraining for Clinically Significant Prostate Cancer Detection in Bi-parametric MRIcited by 3
  5. 2024Looking Beyond What You See: An Empirical Analysis on Subgroup Intersectional Fairness for Multi-label Chest X-ray Classification Using Social Determinants of Racial Health Inequitiescited by 3
  6. 2025Discovering Hierarchy-Grounded Domains with Adaptive Granularity for Clinical Domain Generalizationcited by 3
  7. 2022AutoMap: Automatic Medical Code Mapping for Clinical Prediction Model Deploymentcited by 2
  8. 2023From Single-Hospital to Multi-Centre Applications: Enhancing the Generalisability of Deep Learning Models for Adverse Event Prediction in the ICUcited by 2
  9. 2023MLHOps: Machine Learning for Healthcare Operationscited by 2
  10. 2023Adapting Machine Learning Diagnostic Models to New Populations Using a Small Amount of Data: Results from Clinical Neurosciencecited by 2
  11. 2021Deploying clinical machine learning? Consider the following...cited by 1
  12. 2024Generalization vs. Specialization under Concept Shiftcited by 1
  13. 2025M3DA: Benchmark for Unsupervised Domain Adaptation in 3D Medical Image Segmentationcited by 1
  14. 2023Rescuing referral failures during automated diagnosis of domain-shifted medical imagescited by 0
  15. 2023Towards Context-Aware Domain Generalization: Understanding the Benefits and Limits of Marginal Transfer Learningcited by 0
  16. 2024Out-of-distribution robustness for multivariate analysis via causal regularisationcited by 0
  17. 2025Robust Predictive Modeling Under Unseen Data Distribution Shifts: A Methodological Commentarycited by 0
  18. 2026Causal Transfer in Medical Image Analysiscited by 0
  19. 2026Neuron Incidence Redistribution for Fairness in Medical Image Classificationcited by 0
  20. 2026RobustSeiz: An Open-Source Framework for Benchmarking the Robustness of EEG Seizure Detection Modelscited by 0