mlsanityData to 7 Oct 2026

Why Do Probabilistic Clinical Models Fail To Transport Between Sites?

arXiv 2311.04787 ↗First posted 8 Nov 2023cs.LG cs.PF stat.MLcited by 9

Not on the map. Why?

Abstract (arXiv)

The rising popularity of artificial intelligence in healthcare is highlighting the problem that a computational model achieving super-human clinical performance at its training sites may perform substantially worse at new sites. In this perspective, we present common sources for this failure to transport, which we divide into sources under the control of the experimenter and sources inherent to the clinical data-generating process. Of the inherent sources we look a little deeper into site-specific clinical practices that can affect the data distribution, and propose a potential solution intended to isolate the imprint of those practices on the data from the patterns of disease cause and effect that are the usual target of probabilistic clinical models.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 35 of its 113 references that are in the map. First posted on arXiv 8 Nov 2023.

Builds on

35 of 113 references.

References · 35

  1. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  2. 2019Invariant Risk Minimizationcited by 1,832
  3. 2016Concrete Problems in AI Safetycited by 1,582
  4. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  5. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 919
  6. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 762
  7. 2021Domain Generalization: A Surveycited by 760
  8. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 488
  9. 2017CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learningcited by 459
  10. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 431
  11. 2018Confounding variables can degrade generalization performance of radiological deep learning modelscited by 374
  12. 2017High-dimensional dynamics of generalization error in neural networkscited by 238
23 more
  1. 2019Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imagingcited by 215
  2. 2022Neural Networks and the Chomsky Hierarchycited by 214
  3. 2019Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathologycited by 172
  4. 2021Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolationcited by 130
  5. 2017Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributionscited by 125
  6. 2013Stabilitycited by 119
  7. 2018The Blessings of Multiple Causescited by 103
  8. 2020A causal view of compositional zero-shot recognitioncited by 78
  9. 2019Veridical Data Sciencecited by 65
  10. 2020Triple descent and the two kinds of overfitting: Where & why do they appear?cited by 64
  11. 2018Deep Learning Predicts Hip Fracture using Confounding Patient and Healthcare Variablescited by 50
  12. 2019Feature Robustness in Non-stationary Health Records: Caveats to Deployable Model Performance in Common Clinical Machine Learning Taskscited by 39
  13. 2022Identifying Patient-Specific Root Causes with the Heteroscedastic Noise Modelcited by 38
  14. 2021A Review of Generalizability and Transportabilitycited by 36
  15. 2017Fast Causal Inference with Non-Random Missingness by Test-Wise Deletioncited by 22
  16. 2019A Unifying Causal Framework for Analyzing Dataset Shift-stable Learning Algorithmscited by 17
  17. 2020Accounting for Unobserved Confounding in Domain Generalizationcited by 14
  18. 2022Identifying Patient-Specific Root Causes of Diseasecited by 8
  19. 2022Overcoming Shortcut Learning in a Target Domain by Generalizing Basic Visual Factors from a Source Domaincited by 7
  20. 2020Counterexamples to "The Blessings of Multiple Causes" by Wang and Bleicited by 6
  21. 2022Sample-Specific Root Causal Inference with Latent Variablescited by 6
  22. 2020I-SPEC: An End-to-End Framework for Learning Transportable, Shift-Stable Modelscited by 3
  23. 2023Dynamic Updating of Clinical Survival Prediction Models in a Rapidly Changing Environmentcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 4; 2026: 4.

9 papers.

  1. 2024MetaWearS: A Shortcut in Wearable Systems Lifecycle with Only a Few Shotscited by 2
  2. 2026MedCL-Bench: Benchmarking stability-efficiency trade-offs and scaling in biomedical continual learningcited by 2
  3. 2026Keeping an Eye on AI: A Framework for Effective Human Oversight of AI Systemscited by 2
  4. 2025Cohort-Aware Agents for Individualized Lung Cancer Risk Prediction Using a Retrieval-Augmented Model Selection Frameworkcited by 1
  5. 2025Bridging the Generalisation Gap: Synthetic Data Generation for Multi-Site Clinical Model Validationcited by 0
  6. 2025Cryptogenic stroke and migraine: using probabilistic independence and machine learning to uncover latent sources of disease from the electronic health recordcited by 0
  7. 2025A tutorial on discovering and quantifying the effect of latent causal sources of multimodal EHR datacited by 0
  8. 2026Domain-Adaptive Arrhythmia Classification Using a Hybrid Transformer on Wearable Heart Signalscited by 0
  9. 2026Unveiling Invariant and Transferable Latent Factors Across Heterogeneous Environments via ATLAScited by 0