mlsanityData to 14 Oct 2026

Considerations for Distribution Shift Robustness of Diagnostic Models in Healthcare

arXiv 2410.19575 ↗First posted 25 Oct 2024stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We consider robustness to distribution shifts in the context of diagnostic models in healthcare, where the prediction target Y, e.g., the presence of a disease, is causally upstream of the observations X, e.g., a biomarker. Distribution shifts may occur, for instance, when the training data is collected in a domain with patients having particular demographic characteristics while the model is deployed on patients from a different demographic group. In the domain of applied ML for health, it is common to predict Y from X without considering further information about the patient. However, beyond the direct influence of the disease Y on biomarker X, a predictive model may learn to exploit confounding dependencies (or shortcuts) between X and Y that are unstable under certain distribution shifts. In this work, we highlight a data generating mechanism common to healthcare settings and discuss how recent theoretical results from the causality literature can be applied to build robust predictive models. We theoretically show why ignoring covariates as well as common invariant learning approaches will in general not yield robust predictors in the studied setting, while including certain covariates into the prediction model will. In an extensive simulation study, we showcase the robustness (or lack thereof) of different predictors under various data generating processes. Lastly, we analyze the performance of the different approaches using the PTB-XL dataset, a public dataset of annotated ECG recordings.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 45 references that are in the map. First posted on arXiv 25 Oct 2024.

Builds on

22 of 45 references.

References · 22

  1. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  2. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  3. 2020Shortcut Learning in Deep Neural Networkscited by 1,969
  4. 2019Invariant Risk Minimizationcited by 1,854
  5. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,375
  6. 2017Counterfactual Fairnesscited by 804
  7. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 742
  8. 2015Causal inference using invariant prediction: identification and confidence intervalscited by 536
  9. 2017CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learningcited by 461
  10. 2017Avoiding Discrimination through Causal Reasoningcited by 194
  11. 2020CLOCS: Contrastive Learning of Cardiac Signals Across Space, Time, and Patientscited by 171
  12. 2020Deep Learning for ECG Analysis: Benchmarks and Insights from PTB-XLcited by 158
10 more
  1. 2017Invariant Causal Prediction for Nonlinear Modelscited by 140
  2. 2017Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributionscited by 126
  3. 2018Anchor regression: heterogeneous data meets causalitycited by 118
  4. 2021Self-supervised representation learning from 12-lead ECG datacited by 80
  5. 2021Causally motivated Shortcut Removal Using Auxiliary Labelscited by 55
  6. 2021Out-of-distribution Generalization in the Presence of Nuisance-Induced Spurious Correlationscited by 41
  7. 2019Stabilizing Variable Selection and Regressioncited by 31
  8. 2022Write It Like You See It: Detectable Differences in Clinical Notes By Race Lead To Differential Model Recommendationscited by 20
  9. 2019A Unifying Causal Framework for Analyzing Dataset Shift-stable Learning Algorithmscited by 17
  10. 2022When Personalization Harms: Reconsidering the Use of Group Attributes in Predictioncited by 6

Built on by

Not cited yet.