mlsanityData to 7 Oct 2026

Source Matters: Source Dataset Impact on Model Robustness in Medical Imaging

arXiv 2403.04484 ↗First posted 7 Mar 2024cs.CV cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

Transfer learning has become an essential part of medical imaging classification algorithms, often leveraging ImageNet weights. The domain shift from natural to medical images has prompted alternatives such as RadImageNet, often showing comparable classification performance. However, it remains unclear whether the performance gains from transfer learning stem from improved generalization or shortcut learning. To address this, we conceptualize confounders by introducing the Medical Imaging Contextualized Confounder Taxonomy (MICCAT) and investigate a range of confounders across it -- whether synthetic or sampled from the data -- using two public chest X-ray and CT datasets. We show that ImageNet and RadImageNet achieve comparable classification performance, yet ImageNet is much more prone to overfitting to confounders. We recommend that researchers using ImageNet-pretrained models reexamine their model robustness by conducting similar experiments. Our code and experiments are available at https://github.com/DovileDo/source-matters.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 21 of its 33 references that are in the map. First posted on arXiv 7 Mar 2024.

Builds on

21 of 33 references.

References · 21

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Densely Connected Convolutional Networkscited by 6,691
  3. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  4. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  5. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,509
  6. 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 969
  7. 2018Confounding variables can degrade generalization performance of radiological deep learning modelscited by 374
  8. 2019Using Pre-Training Can Improve Model Robustness and Uncertaintycited by 373
  9. 2019Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imagingcited by 215
  10. 2021Physics-based Noise Modeling for Extreme Low-light Photographycited by 67
  11. 2022What Makes Transfer Learning Work For Medical Images: Feature Reuse & Other Factorscited by 61
  12. 2022A Data-Based Perspective on Transfer Learningcited by 32
9 more
  1. 2023On the Connection between Pre-training Data Diversity and Fine-tuning Robustnesscited by 25
  2. 2023UniChest: Conquer-and-Divide Pre-training for Multi-Source Chest X-Ray Classificationcited by 21
  3. 2023The Role of Pre-training Data in Transfer Learningcited by 17
  4. 2023Right for the Wrong Reason: Can Interpretable ML Techniques Detect Spurious Correlations?cited by 16
  5. 2018Cats or CAT scans: transfer learning from natural or medical image source datasets?cited by 14
  6. 2020Deep Learning for Distinguishing Normal versus Abnormal Chest Radiographs and Generalization to Unseen Diseasescited by 12
  7. 2023Robustness Stress Testing in Medical Image Classificationcited by 5
  8. 2022Does Medical Imaging learn different Convolution Filters?cited by 2
  9. 2023Revisiting Hidden Representations in Transfer Learning for Medical Imagingcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 6; 2026: 1.

10 papers.

  1. 2025In the Picture: Medical Imaging Datasets, Artifacts, and their Living Reviewcited by 8
  2. 2024Mask of truth: model sensitivity to unexpected regions of medical imagescited by 6
  3. 2025General Methods Make Great Domain-specific Foundation Models: A Case-study on Fetal Ultrasoundcited by 6
  4. 2025In-hoc Concept Representations to Regularise Deep Learning in Medical Imagingcited by 2
  5. 2024Exploring connections of spectral analysis and transfer learning in medical imagingcited by 1
  6. 2024SeagrassFinder: Deep Learning for Eelgrass Detection and Coverage Estimation in the Wildcited by 1
  7. 2025Revisiting Performance Claims for Chest X-Ray Models Using Clinical Contextcited by 1
  8. 2025Learning to Harmonize Cross-vendor X-ray Images by Non-linear Image Dynamics Correctioncited by 0
  9. 2025Mitigating Overfitting in Medical Imaging: Self-Supervised Pretraining vs. ImageNet Transfer Learning for Dermatological Diagnosiscited by 0
  10. 2026Look What the Probes Dragged In! Real-World Chest X-ray Shortcuts in MedCLIPcited by 0