mlsanityData to 7 Oct 2026

Robust and Efficient Medical Imaging with Self-Supervision

arXiv 2205.09723 ↗First posted 19 May 2022cs.CV cs.AI cs.LGcited by 40

Not on the map. Why?

Abstract (arXiv)

Recent progress in Medical Artificial Intelligence (AI) has delivered systems that can reach clinical expert level performance. However, such systems tend to demonstrate sub-optimal "out-of-distribution" performance when evaluated in clinical settings different from the training environment. A common mitigation strategy is to develop separate systems for each clinical setting using site-specific data [1]. However, this quickly becomes impractical as medical data is time-consuming to acquire and expensive to annotate [2]. Thus, the problem of "data-efficient generalization" presents an ongoing difficulty for Medical AI development. Although progress in representation learning shows promise, their benefits have not been rigorously studied, specifically for out-of-distribution settings. To meet these challenges, we present REMEDIS, a unified representation learning strategy to improve robustness and data-efficiency of medical imaging AI. REMEDIS uses a generic combination of large-scale supervised transfer learning with self-supervised learning and requires little task-specific customization. We study a diverse range of medical imaging tasks and simulate three realistic application scenarios using retrospective data. REMEDIS exhibits significantly improved in-distribution performance with up to 11.5% relative improvement in diagnostic accuracy over a strong supervised baseline. More importantly, our strategy leads to strong data-efficient generalization of medical imaging AI, matching strong supervised baselines using between 1% to 33% of retraining data across tasks. These results suggest that REMEDIS can significantly accelerate the life-cycle of medical imaging AI development thereby presenting an important step forward for medical imaging AI to deliver broad impact.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). Builds on 79 of its 136 references that are in the map. First posted on arXiv 19 May 2022.

Builds on

79 of 136 references.

References · 79

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  6. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  7. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  9. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  10. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  11. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  12. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
38 more
  1. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  2. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  3. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  4. 2018Attention-based Deep Multiple Instance Learningcited by 1,077
  5. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  6. 2019EfficientDet: Scalable and Efficient Object Detectioncited by 1,018
  7. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 916
  8. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 806
  9. 2021Robust fine-tuning of zero-shot modelscited by 785
  10. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 762
  11. 2021Domain Generalization: A Surveycited by 760
  12. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 733
  13. 2018Deep Visual Domain Adaptation: A Surveycited by 634
  14. 2013Domain Generalization via Invariant Feature Representationcited by 622
  15. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 604
  16. 2017Large Batch Training of Convolutional Networkscited by 529
  17. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 488
  18. 2018Confounding variables can degrade generalization performance of radiological deep learning modelscited by 374
  19. 2019Using Pre-Training Can Improve Model Robustness and Uncertaintycited by 373
  20. 2019Transfusion: Understanding Transfer Learning for Medical Imagingcited by 366
  21. 2020What is being transferred in transfer learning?cited by 365
  22. 2017Unified Deep Supervised Domain Adaptation and Generalizationcited by 322
  23. 2021Big Self-Supervised Models Advance Medical Image Classificationcited by 320
  24. 2020Contrastive learning of global and local features for medical image segmentation with limited annotationscited by 309
  25. 2020Pretrained Transformers Improve Out-of-Distribution Robustnesscited by 293
  26. 2020Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depthcited by 241
  27. 2019Unsupervised Embedding Learning via Invariant and Spreading Instance Featurecited by 231
  28. 2020A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promisescited by 230
  29. 2018Characterizing and Avoiding Negative Transfercited by 226
  30. 2020How Well Do Self-Supervised Models Transfer?cited by 213
  31. 2021Towards Causal Representation Learningcited by 207
  32. 2017Multi-task Self-Supervised Visual Learningcited by 200
  33. 2020Representation Learning via Invariant Causal Mechanismscited by 197
  34. 2021Self-supervised Pretraining of Visual Features in the Wildcited by 197
  35. 2016What makes ImageNet good for transfer learning?cited by 192
  36. 2020On the Theory of Transfer Learning: The Importance of Task Diversitycited by 186
  37. 2020Contrastive Training for Improved Out-of-Distribution Detectioncited by 186
  38. 2017Colorization as a Proxy Task for Visual Understandingcited by 175

50 most cited of 79 All 79 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 9; 2023: 14; 2024: 11; 2025: 3; 2026: 3.

40 papers.

Sort:

By area:

  1. 2023Generative models improve fairness of medical classifiers under distribution shiftscited by 90
  2. 2023MOTOR: A Time-To-Event Foundation Model For Structured Medical Recordscited by 75
  3. 2022MEDFAIR: Benchmarking Fairness for Medical Imagingcited by 66
  4. 2023Parameter-Efficient Fine-Tuning for Medical Image Analysis: The Missed Opportunitycited by 54
  5. 2024Towards a clinically accessible radiology foundation model: open-access and lightweight, with automated evaluationcited by 51
  6. 2022SVL-Adapter: Self-Supervised Adapter for Vision-Language Pretrained Modelscited by 34
  7. 2024How to build the best medical image segmentation algorithm using foundation models: a comprehensive empirical study with Segment Anything Modelcited by 32
  8. 2024Review of multimodal machine learning approaches in healthcarecited by 30
  9. 2022Risk of Bias in Chest Radiography Deep Learning Foundation Modelscited by 26
  10. 2022How Robust is Unsupervised Representation Learning to Distribution Shift?cited by 24
  11. 2023PECoP: Parameter Efficient Continual Pretraining for Action Quality Assessmentcited by 19
  12. 2023Exploring Image Augmentations for Siamese Representation Learning with Chest X-Rayscited by 12
28 more
  1. 2022Self-Supervised Learning with Limited Labeled Data for Prostate Cancer Detection in High Frequency Ultrasoundcited by 7
  2. 2023Large-scale pretraining on pathological images for fine-tuning of small pathological benchmarkscited by 7
  3. 2023Evaluating AI systems under uncertain ground truth: a case study in dermatologycited by 7
  4. 2024Time-to-Event Pretraining for 3D Medical Imagingcited by 7
  5. 2023Video Pretraining Advances 3D Deep Learning on Chest CT Taskscited by 4
  6. 2024Learning Low-Rank Feature for Thorax Disease Classificationcited by 4
  7. 2023Cross-Shaped Windows Transformer with Self-supervised Pretraining for Clinically Significant Prostate Cancer Detection in Bi-parametric MRIcited by 3
  8. 2024ConPro: Learning Severity Representation for Medical Images using Contrastive Learning and Preference Optimizationcited by 3
  9. 2025Towards disentangling the contributions of articulation and acoustics in multimodal phoneme recognitioncited by 3
  10. 2026MLLM-HWSI: A Multimodal Large Language Model for Hierarchical Whole Slide Image Understandingcited by 3
  11. 2022Federated Training of Dual Encoding Models on Small Non-IID Client Datasetscited by 2
  12. 2023BenchMD: A Benchmark for Unified Learning on Medical Images and Sensorscited by 2
  13. 2023FBA-Net: Foreground and Background Aware Contrastive Learning for Semi-Supervised Atrium Segmentationcited by 2
  14. 2023Revisiting Fine-Tuning Strategies for Self-supervised Medical Imaging Analysiscited by 2
  15. 2024Embedding-based Multimodal Learning on Pan-Squamous Cell Carcinomas for Improved Survival Outcomescited by 2
  16. 2022Robust Alzheimer's Progression Modeling using Cross-Domain Self-Supervised Deep Learningcited by 1
  17. 2023MedLSAM: Localize and Segment Anything Model for 3D CT Imagescited by 1
  18. 2026From pre-training to downstream performance: Does domain-specific pre-training make sense?cited by 1
  19. 2022Joint Debiased Representation and Image Clustering Learning with Self-Supervisioncited by 0
  20. 2022Calibrating Ensembles for Scalable Uncertainty Quantification in Deep Learning-based Medical Segmentationcited by 0
  21. 2023How to Train Your CheXDragon: Training Chest X-Ray Models for Transfer to Novel Tasks and Healthcare Systemscited by 0
  22. 2024Exploiting Liver CT scans in Colorectal Carcinoma genomics mutation classificationcited by 0
  23. 2024Self-Supervised Learning for Building Robust Pediatric Chest X-ray Classification Modelscited by 0
  24. 2024Deep Learning for Classification of Inflammatory Bowel Disease Activity in Whole Slide Images of Colonic Histopathologycited by 0
  25. 2024Enhancing Skin Lesion Classification Generalization with Active Domain Adaptationcited by 0
  26. 2025The Next Layer: Augmenting Foundation Models with Structure-Preserving and Attention-Guided Learning for Local Patches to Global Context Awareness in Computational Pathologycited by 0
  27. 2025Harmonizing Generalization and Specialization: Uncertainty-Informed Collaborative Learning for Semi-supervised Medical Image Segmentationcited by 0
  28. 2026From Multi-Resolution Cells to Gigapixel Whole Slide Images Foundation Model for Computational Pathologycited by 0