mlsanityData to 7 Oct 2026

Dive into Self-Supervised Learning for Medical Image Analysis: Data, Models and Tasks

arXiv 2209.12157 ↗First posted 25 Sep 2022cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Self-supervised learning (SSL) has achieved remarkable performance in various medical imaging tasks by dint of priors from massive unlabelled data. However, regarding a specific downstream task, there is still a lack of an instruction book on how to select suitable pretext tasks and implementation details throughout the standard ``pretrain-then-finetune'' workflow. In this work, we focus on exploiting the capacity of SSL in terms of four realistic and significant issues: (1) the impact of SSL on imbalanced datasets, (2) the network architecture, (3) the applicability of upstream tasks to downstream tasks and (4) the stacking effect of SSL and common policies for deep learning. We provide a large-scale, in-depth and fine-grained study through extensive experiments on predictive, contrastive, generative and multi-SSL algorithms. Based on the results, we have uncovered several insights. Positively, SSL advances class-imbalanced learning mainly by boosting the performance of the rare class, which is of interest to clinical diagnosis. Unfortunately, SSL offers marginal or even negative returns in some cases, including severely imbalanced and relatively balanced data regimes, as well as combinations with common training policies. Our intriguing findings provide practical guidelines for the usage of SSL in the medical context and highlight the need for developing universal pretext tasks to accommodate diverse application scenarios.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 72 of its 104 references that are in the map. First posted on arXiv 25 Sep 2022.

Builds on

72 of 104 references.

References · 72

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  8. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  9. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  10. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  11. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  12. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
38 more
  1. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  2. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  3. 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,471
  4. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  5. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  6. 2016Colorful Image Colorizationcited by 1,113
  7. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  8. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
  9. 2014Deep Learning in Neural Networks: An Overviewcited by 765
  10. 2020Self-supervised Learning: Generative or Contrastivecited by 693
  11. 2021The Medical Segmentation Decathloncited by 620
  12. 2019A Theoretical Analysis of Contrastive Unsupervised Representation Learningcited by 530
  13. 2019The Liver Tumor Segmentation Benchmark (LiTS)cited by 504
  14. 2019Transfusion: Understanding Transfer Learning for Medical Imagingcited by 366
  15. 2020What is being transferred in transfer learning?cited by 365
  16. 2020Rethinking Pre-training and Self-trainingcited by 352
  17. 2020Contrastive learning of global and local features for medical image segmentation with limited annotationscited by 309
  18. 2016Revisiting Batch Normalization For Practical Domain Adaptationcited by 285
  19. 2020Rethinking the Value of Labels for Improving Class-Imbalanced Learningcited by 216
  20. 2020How Well Do Self-Supervised Models Transfer?cited by 213
  21. 2019Domain-Specific Batch Normalization for Unsupervised Domain Adaptationcited by 197
  22. 2017Colorization as a Proxy Task for Visual Understandingcited by 175
  23. 2015DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentationcited by 158
  24. 20203D Self-Supervised Methods for Medical Imagingcited by 134
  25. 2014Hypercolumns for Object Segmentation and Fine-grained Localizationcited by 128
  26. 2020Self-Supervision with Superpixels: Training Few-shot Medical Image Segmentation without Annotationcited by 128
  27. 2016The Importance of Skip Connections in Biomedical Image Segmentationcited by 126
  28. 2021Self-supervised Learning is More Robust to Dataset Imbalancecited by 122
  29. 2020What makes instance discrimination good for transfer learning?cited by 111
  30. 2021Recent advances and clinical applications of deep learning in medical image analysiscited by 104
  31. 2021A Broad Study on the Transferability of Visual Representations with Contrastive Learningcited by 78
  32. 2019Self-supervised Feature Learning for 3D Medical Images by Playing a Rubik's Cubecited by 76
  33. 2021Self-supervised learning methods and applications in medical imaging analysis: A surveycited by 76
  34. 2019A critical analysis of self-supervision, or what we can learn from a single imagecited by 75
  35. 2019Self-Supervised Learning for Cardiac MR Image Segmentation by Anatomical Position Predictioncited by 70
  36. 2020Comparing to Learn: Surpassing ImageNet Pretraining on Radiographs By Comparing Image Representationscited by 69
  37. 2021Preservational Learning Improves Self-supervised Medical Image Models by Reconstructing Diverse Contextscited by 59
  38. 2021MedAug: Contrastive learning leveraging patient metadata improves representations for chest X-ray interpretationcited by 52

50 most cited of 72 All 72 (JSON)

Built on by

Not cited yet.