mlsanityData to 7 Oct 2026

A Review of Predictive and Contrastive Self-supervised Learning for Medical Images

arXiv 2302.05043 ↗First posted 10 Feb 2023cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

Over the last decade, supervised deep learning on manually annotated big data has been progressing significantly on computer vision tasks. But the application of deep learning in medical image analysis was limited by the scarcity of high-quality annotated medical imaging data. An emerging solution is self-supervised learning (SSL), among which contrastive SSL is the most successful approach to rivalling or outperforming supervised learning. This review investigates several state-of-the-art contrastive SSL algorithms originally on natural images as well as their adaptations for medical images, and concludes by discussing recent advances, current limitations, and future directions in applying contrastive SSL in the medical domain.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 142 of its 243 references that are in the map. First posted on arXiv 10 Feb 2023.

Builds on

142 of 243 references.

References · 142

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  8. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  9. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  10. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  11. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  12. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
38 more
  1. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  2. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  3. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  4. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  5. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
  6. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  7. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  8. 2019Contrastive Multiview Codingcited by 1,297
  9. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  10. 2016Colorful Image Colorizationcited by 1,113
  11. 2018Federated Learning with Non-IID Datacited by 1,099
  12. 2018Attention-based Deep Multiple Instance Learningcited by 1,077
  13. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  14. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,020
  15. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 806
  16. 2015Unsupervised Deep Embedding for Clustering Analysiscited by 763
  17. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 733
  18. 2020Self-supervised Learning: Generative or Contrastivecited by 693
  19. 2018MINE: Mutual Information Neural Estimationcited by 680
  20. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 672
  21. 2019Self-supervised Visual Feature Learning with Deep Neural Networks: A Surveycited by 656
  22. 2020Contrastive Learning of Medical Visual Representations from Paired Images and Textcited by 636
  23. 2017Wasserstein GANcited by 635
  24. 2020A Survey on Contrastive Self-supervised Learningcited by 553
  25. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 504
  26. 2020Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learningcited by 473
  27. 2017CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learningcited by 459
  28. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
  29. 2020Hard Negative Mixing for Contrastive Learningcited by 399
  30. 2018Rethinking ImageNet Pre-trainingcited by 376
  31. 2019Transfusion: Understanding Transfer Learning for Medical Imagingcited by 366
  32. 2021Big Self-Supervised Models Advance Medical Image Classificationcited by 320
  33. 2019On Mutual Information Maximization for Representation Learningcited by 315
  34. 2020Contrastive learning of global and local features for medical image segmentation with limited annotationscited by 309
  35. 2019Unsupervised Embedding Learning via Invariant and Spreading Instance Featurecited by 231
  36. 2020What Should Not Be Contrastive in Contrastive Learningcited by 224
  37. 2017Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?cited by 220
  38. 2017Unsupervised Representation Learning by Sorting Sequencescited by 219

50 most cited of 142 All 142 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 2; 2025: 0; 2026: 4.

10 papers.

  1. 2023UniPAD: A Universal Pre-training Paradigm for Autonomous Drivingcited by 73
  2. 2023Generalizable Sleep Staging via Multi-Level Domain Alignmentcited by 17
  3. 2023Revisiting Computer-Aided Tuberculosis Diagnosiscited by 6
  4. 2023Echocardiogram Foundation Model -- Application 1: Estimating Ejection Fractioncited by 1
  5. 2024From Uncertainty to Clarity: Uncertainty-Guided Class-Incremental Learning for Limited Biomedical Samples via Semantic Expansioncited by 1
  6. 2024Self-Supervised Learning Featuring Small-Scale Image Dataset for Treatable Retinal Diseases Classificationcited by 0
  7. 2026Cardiac Output Prediction from Echocardiograms: Self-Supervised Learning with Limited Datacited by 0
  8. 2026Spectral Vision Transformer for Efficient Tokenization with Limited Datacited by 0
  9. 2026Vision Foundation Models in Radiology: A Scoping Review of Data, Methodology, Evaluation and Clinical Translationcited by 0
  10. 2026Domain-Specific Self-Supervised Representation Learning for Retinal Fundus Classificationcited by 0