# An Empirical Framework for Domain Generalization in Clinical Settings

arXiv: https://arxiv.org/abs/2103.11163 · HTML: https://mlsanity.com/paper/2103.11163/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 59 of its 87 references that are in the map.
- First posted on arXiv 20 Mar 2021.

## Builds on (most cited first, 30 of 59)

- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases](https://mlsanity.com/paper/1705.02315/) (2017)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs](https://mlsanity.com/paper/1901.07042/) (2019)
- [Causal inference using invariant prediction: identification and confidence intervals](https://mlsanity.com/paper/1501.01332/) (2015)
- [Domain Generalization by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1903.06864/) (2019)
- [Domain Generalization via Model-Agnostic Learning of Semantic Features](https://mlsanity.com/paper/1910.13580/) (2019)
- [Confounding variables can degrade generalization performance of radiological deep learning models](https://mlsanity.com/paper/1807.00431/) (2018)
- [Transfusion: Understanding Transfer Learning for Medical Imaging](https://mlsanity.com/paper/1902.07208/) (2019)
- [PadChest: A large chest x-ray image dataset with multi-label annotated reports](https://mlsanity.com/paper/1901.07441/) (2019)
- [Scalable and accurate deep learning for electronic health records](https://mlsanity.com/paper/1801.07860/) (2018)
- [The Risks of Invariant Risk Minimization](https://mlsanity.com/paper/2010.05761/) (2020)
- [Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imaging](https://mlsanity.com/paper/1909.12475/) (2019)
- [Causality matters in medical imaging](https://mlsanity.com/paper/1912.08142/) (2019)
- [Invariant Risk Minimization Games](https://mlsanity.com/paper/2002.04692/) (2020)
- [CheXclusion: Fairness gaps in deep chest X-ray classifiers](https://mlsanity.com/paper/2003.00827/) (2020)
- [Invariant Causal Prediction for Nonlinear Models](https://mlsanity.com/paper/1706.08576/) (2017)
- [Efficient Domain Generalization via Common-Specific Low-Rank Decomposition](https://mlsanity.com/paper/2003.12815/) (2020)

## Built on by (most cited first, 30 of 32)

- [Diagnosing failures of fairness transfer across distribution shift in real-world medical settings](https://mlsanity.com/paper/2202.01034/) (2022)
- [Robust and Efficient Medical Imaging with Self-Supervision](https://mlsanity.com/paper/2205.09723/) (2022)
- [WOODS: Benchmarks for Out-of-Distribution Generalization in Time Series](https://mlsanity.com/paper/2203.09978/) (2022)
- [ManyDG: Many-domain Generalization for Healthcare Applications](https://mlsanity.com/paper/2301.08834/) (2023)
- [A Review of Causality for Learning Algorithms in Medical Image Analysis](https://mlsanity.com/paper/2206.05498/) (2022)
- ["Why did the Model Fail?": Attributing Model Performance Changes to Distribution Shifts](https://mlsanity.com/paper/2210.10769/) (2022)
- [When More is Less: Incorporating Additional Datasets Can Hurt Performance By Introducing Spurious Correlations](https://mlsanity.com/paper/2308.04431/) (2023)
- [Evaluating Latent Space Robustness and Uncertainty of EEG-ML Models under Realistic Distribution Shifts](https://mlsanity.com/paper/2209.11233/) (2022)
- [A comparison of approaches to improve worst-case predictive model performance over patient subpopulations](https://mlsanity.com/paper/2108.12250/) (2021)
- [Domain generalization in deep learning-based mass detection in mammography: A large-scale multi-center study](https://mlsanity.com/paper/2201.11620/) (2022)
- [A Systematic Review of Generalization Research in Medical Image Classification](https://mlsanity.com/paper/2403.12167/) (2024)
- [DSLOB: A Synthetic Limit Order Book Dataset for Benchmarking Forecasting Algorithms under Distributional Shift](https://mlsanity.com/paper/2211.11513/) (2022)
- [The Limits of Fair Medical Imaging AI In The Wild](https://mlsanity.com/paper/2312.10083/) (2023)
- [PhysioMTL: Personalizing Physiological Patterns using Optimal Transport Multi-Task Regression](https://mlsanity.com/paper/2203.12595/) (2022)
- [Looking for Out-of-Distribution Environments in Multi-center Critical Care Data](https://mlsanity.com/paper/2205.13398/) (2022)
- [Cross-Shaped Windows Transformer with Self-supervised Pretraining for Clinically Significant Prostate Cancer Detection in Bi-parametric MRI](https://mlsanity.com/paper/2305.00385/) (2023)
- [Looking Beyond What You See: An Empirical Analysis on Subgroup Intersectional Fairness for Multi-label Chest X-ray Classification Using Social Determinants of Racial Health Inequities](https://mlsanity.com/paper/2403.18196/) (2024)
- [Discovering Hierarchy-Grounded Domains with Adaptive Granularity for Clinical Domain Generalization](https://mlsanity.com/paper/2506.06977/) (2025)
- [AutoMap: Automatic Medical Code Mapping for Clinical Prediction Model Deployment](https://mlsanity.com/paper/2203.02446/) (2022)
- [From Single-Hospital to Multi-Centre Applications: Enhancing the Generalisability of Deep Learning Models for Adverse Event Prediction in the ICU](https://mlsanity.com/paper/2303.15354/) (2023)
- [MLHOps: Machine Learning for Healthcare Operations](https://mlsanity.com/paper/2305.02474/) (2023)
- [Adapting Machine Learning Diagnostic Models to New Populations Using a Small Amount of Data: Results from Clinical Neuroscience](https://mlsanity.com/paper/2308.03175/) (2023)
- [Deploying clinical machine learning? Consider the following...](https://mlsanity.com/paper/2109.06919/) (2021)
- [Generalization vs. Specialization under Concept Shift](https://mlsanity.com/paper/2409.15582/) (2024)
- [M3DA: Benchmark for Unsupervised Domain Adaptation in 3D Medical Image Segmentation](https://mlsanity.com/paper/2502.17029/) (2025)
- [Rescuing referral failures during automated diagnosis of domain-shifted medical images](https://mlsanity.com/paper/2311.16766/) (2023)
- [Towards Context-Aware Domain Generalization: Understanding the Benefits and Limits of Marginal Transfer Learning](https://mlsanity.com/paper/2312.10107/) (2023)
- [Out-of-distribution robustness for multivariate analysis via causal regularisation](https://mlsanity.com/paper/2403.01865/) (2024)
- [Robust Predictive Modeling Under Unseen Data Distribution Shifts: A Methodological Commentary](https://mlsanity.com/paper/2503.03399/) (2025)
- [Causal Transfer in Medical Image Analysis](https://mlsanity.com/paper/2603.24388/) (2026)

## Abstract (arXiv)

Clinical machine learning models experience significantly degraded performance in datasets not seen during training, e.g., new hospitals or populations. Recent developments in domain generalization offer a promising solution to this problem by creating models that learn invariances across environments. In this work, we benchmark the performance of eight domain generalization methods on multi-site clinical time series and medical imaging data. We introduce a framework to induce synthetic but realistic domain shifts and sampling bias to stress-test these methods over existing non-healthcare benchmarks. We find that current domain generalization methods do not consistently achieve significant gains in out-of-distribution performance over empirical risk minimization on real-world medical imaging data, in line with prior work on general imaging datasets. However, a subset of realistic induced-shift scenarios in clinical time series data do exhibit limited performance gains. We characterize these scenarios in detail, and recommend best practices for domain generalization in the clinical setting.