mlsanityData to 7 Oct 2026

Supervised Transfer Learning at Scale for Medical Imaging

arXiv 2101.05913 ↗First posted 14 Jan 2021cs.CVcited by 38

Not on the map. Why?

Abstract (arXiv)

Transfer learning is a standard technique to improve performance on tasks with limited data. However, for medical imaging, the value of transfer learning is less clear. This is likely due to the large domain mismatch between the usual natural-image pre-training (e.g. ImageNet) and medical images. However, recent advances in transfer learning have shown substantial improvements from scale. We investigate whether modern methods can change the fortune of transfer learning for medical imaging. For this, we study the class of large-scale pre-trained networks presented by Kolesnikov et al. on three diverse imaging tasks: chest radiography, mammography, and dermatology. We study both transfer performance and critical properties for the deployment in the medical domain, including: out-of-distribution generalization, data-efficiency, sub-group fairness, and uncertainty estimation. Interestingly, we find that for some of these properties transfer from natural to medical images is indeed extremely effective, but only when performed at sufficient scale.

Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). Builds on 40 of its 57 references that are in the map. First posted on arXiv 14 Jan 2021.

Builds on

40 of 57 references.

References · 40

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2017On Calibration of Modern Neural Networkscited by 4,860
  6. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  7. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  8. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  9. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  10. 2018Group Normalizationcited by 1,514
  11. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
  12. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
28 more
  1. 2014How transferable are features in deep neural networks?cited by 1,216
  2. 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 969
  3. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 916
  4. 2017In Defense of the Triplet Loss for Person Re-Identificationcited by 784
  5. 2019Big Transfer (BiT): General Visual Representation Learningcited by 766
  6. 2017Focal Loss for Dense Object Detectioncited by 749
  7. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 604
  8. 2018Do Better ImageNet Models Transfer Better?cited by 560
  9. 2017Learning multiple visual domains with residual adapterscited by 553
  10. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 488
  11. 2017CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learningcited by 459
  12. 2019A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmarkcited by 419
  13. 2019Transfusion: Understanding Transfer Learning for Medical Imagingcited by 366
  14. 2020What is being transferred in transfer learning?cited by 365
  15. 2019Med3D: Transfer Learning for 3D Medical Image Analysiscited by 282
  16. 2019Billion-scale semi-supervised learning for image classificationcited by 232
  17. 2016What makes ImageNet good for transfer learning?cited by 192
  18. 2015Learning Visual Features from Large Weakly Supervised Datacited by 165
  19. 2019A deep learning system for differential diagnosis of skin diseasescited by 118
  20. 2020Image Representations Learned With Unsupervised Pre-Training Contain Human-like Biasescited by 107
  21. 2020On Robustness and Transferability of Convolutional Neural Networkscited by 98
  22. 2016Learning Visual N-Grams from Web Datacited by 95
  23. 2017Detecting Cancer Metastases on Gigapixel Pathology Imagescited by 51
  24. 2019Micro-Batch Training with Batch-Channel Normalization and Weight Standardizationcited by 45
  25. 2017Knowledge Transfer for Melanoma Screening with Deep Learningcited by 19
  26. 2020CheXpedition: Investigating Generalization Challenges for Translation of Chest X-Ray Algorithms to the Clinical Settingcited by 18
  27. 2020Joint Liver Lesion Segmentation and Classification via Transfer Learningcited by 14
  28. 2019Dual Network Architecture for Few-view CT -- Trained on ImageNet Data and Transferred for Medical Imagingcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 14; 2023: 4; 2024: 4; 2025: 4; 2026: 1.

38 papers.

Sort:

By area:

  1. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 671
  2. 2021Revisiting the Calibration of Modern Neural Networkscited by 387
  3. 2024Exploring scalable medical image encoders beyond text supervisioncited by 129
  4. 2021Exploring the Limits of Large Scale Pre-trainingcited by 104
  5. 2021Does Your Dermatology Classifier Know What It Doesn't Know? Detecting the Long-Tail of Unseen Conditionscited by 68
  6. 2022Medical Image Understanding with Pretrained Vision Language Models: A Comprehensive Studycited by 66
  7. 2022What Makes Transfer Learning Work For Medical Images: Feature Reuse & Other Factorscited by 61
  8. 2021Factors of Influence for Transfer Learning across Diverse Appearance Domains and Task Typescited by 54
  9. 2022A Unifying Theory of Distance from Calibrationcited by 51
  10. 2022Robust and Efficient Medical Imaging with Self-Supervisioncited by 40
  11. 2021A Systematic Benchmarking Analysis of Transfer Learning for Medical Image Analysiscited by 31
  12. 2022How stable are Transferability Metrics evaluations?cited by 30
26 more
  1. 2023Evaluating General Purpose Vision Foundation Models for Medical Image Analysis: An Experimental Study of DINOv2 on Radiology Benchmarkscited by 30
  2. 2021Meta-Learning to Improve Pre-Trainingcited by 27
  3. 2021Transferability Metrics for Selecting Source Model Ensemblescited by 24
  4. 2022Significantly improving zero-shot X-ray pathology classification via fine-tuning pre-trained image-text encoderscited by 15
  5. 2022Deep transfer learning for image classification: a surveycited by 13
  6. 2021Effect of Pre-Training Scale on Intra- and Inter-Domain Full and Few-Shot Transfer Learning for Natural and Medical X-Ray Chest Imagescited by 11
  7. 2022Predicting on the Edge: Identifying Where a Larger Model Does Bettercited by 6
  8. 2022COVID Detection and Severity Prediction with 3D-ConvNeXt and Custom Pretrainingscited by 6
  9. 2022Transfer Learning for Retinal Vascular Disease Detection: A Pilot Study with Diabetic Retinopathy and Retinopathy of Prematuritycited by 5
  10. 2023Building a Winning Team: Selecting Source Model Ensembles using a Submodular Transferability Estimation Approachcited by 5
  11. 2024Scaling Laws for Galaxy Imagescited by 5
  12. 2022Predicting Knee Osteoarthritis Progression from Structural MRI using Deep Learningcited by 4
  13. 2025Is Visual in-Context Learning for Compositional Medical Tasks within Reach?cited by 3
  14. 2022Transfer Learning via Test-Time Neural Networks Aggregationcited by 2
  15. 2021Contrastive Learning Meets Transfer Learning: A Case Study In Medical Image Analysiscited by 1
  16. 2023Parameter-efficient Multi-Task and Multi-Domain Learning using Factorized Tensor Networkscited by 1
  17. 2025APLA: A Simple Adaptation Method for Vision Transformerscited by 1
  18. 2021A new semi-supervised inductive transfer learning framework: Co-Transfercited by 0
  19. 2022Holistic Fine-grained GGS Characterization: From Detection to Unbalanced Classificationcited by 0
  20. 2022Weakly Supervised Learning with Automated Labels from Radiology Reports for Glioma Change Detectioncited by 0
  21. 2023Building RadiologyNET: Unsupervised annotation of a large-scale multimodal medical databasecited by 0
  22. 2024Enhancing Skin Lesion Classification Generalization with Active Domain Adaptationcited by 0
  23. 2024The Evolution and Future Perspectives of Artificial Intelligence Generated Contentcited by 0
  24. 2025Unsupervised Source-Free Ranking of Biomedical Segmentation Models Under Distribution Shiftcited by 0
  25. 2025MetaChest: Generalized few-shot learning of pathologies from chest X-rayscited by 0
  26. 2026The Gait Signature of Frailty: Transfer Learning based Deep Gait Models for Scalable Frailty Assessmentcited by 0