mlsanityData to 7 Oct 2026

An Evaluation of Self-Supervised Pre-Training for Skin-Lesion Analysis

arXiv 2106.09229 ↗First posted 17 Jun 2021cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

Self-supervised pre-training appears as an advantageous alternative to supervised pre-trained for transfer learning. By synthesizing annotations on pretext tasks, self-supervision allows to pre-train models on large amounts of pseudo-labels before fine-tuning them on the target task. In this work, we assess self-supervision for the diagnosis of skin lesions, comparing three self-supervised pipelines to a challenging supervised baseline, on five test datasets comprising in- and out-of-distribution samples. Our results show that self-supervision is competitive both in improving accuracies and in reducing the variability of outcomes. Self-supervision proves particularly useful for low training data scenarios (<1,500 and <150 samples), where its ability to stabilize the outcomes is essential to provide sound results.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 36 of its 49 references that are in the map. First posted on arXiv 17 Jun 2021.

Builds on

36 of 49 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  4. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  5. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  6. 2020Supervised Contrastive Learningcited by 2,803
  7. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  8. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  9. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  10. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  11. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  12. 2016Colorful Image Colorizationcited by 1,113
24 more
  1. 2020What Makes for Good Views for Contrastive Learning?cited by 843
  2. 2020Self-supervised Learning: Generative or Contrastivecited by 693
  3. 2019Self-supervised Visual Feature Learning with Deep Neural Networks: A Surveycited by 656
  4. 2019Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networkscited by 575
  5. 2017Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)cited by 549
  6. 2020CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instancescited by 441
  7. 2021Big Self-Supervised Models Advance Medical Image Classificationcited by 320
  8. 2020A Patient-Centric Dataset of Images and Metadata for Identifying Melanomas Using Clinical Contextcited by 146
  9. 2020PAD-UFES-20: a skin lesion dataset composed of patient data and clinical images collected from smartphonescited by 116
  10. 2020Discriminative Sounding Objects Localization via Self-supervised Audiovisual Matchingcited by 104
  11. 2021When Does Contrastive Visual Representation Learning Work?cited by 92
  12. 2019Self-Supervised Learning for Cardiac MR Image Segmentation by Anatomical Position Predictioncited by 70
  13. 2020Comparing to Learn: Surpassing ImageNet Pretraining on Radiographs By Comparing Image Representationscited by 69
  14. 2021MedAug: Contrastive learning leveraging patient metadata improves representations for chest X-ray interpretationcited by 52
  15. 2020Debiasing Skin Lesion Datasets and Models? Not So Fastcited by 43
  16. 2020Momentum Contrastive Learning for Few-Shot COVID-19 Diagnosis from Chest CT Imagescited by 43
  17. 2020Learning Robust and Multilingual Speech Representationscited by 39
  18. 2021COVID-19 Prognosis via Self-Supervised Representation Learning and Multi-Image Predictioncited by 36
  19. 2019(De)Constructing Bias on Skin Lesion Datasetscited by 30
  20. 2017Self-Supervised Learning for Spinal MRIscited by 24
  21. 2017Knowledge Transfer for Melanoma Screening with Deep Learningcited by 19
  22. 2020Towards Good Practices in Self-supervised Representation Learningcited by 19
  23. 2018Self-supervised Learning for Dense Depth Estimation in Monocular Endoscopycited by 13
  24. 2017Data, Depth, and Design: Learning Reliable Models for Skin Lesion Analysiscited by 8

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 1; 2023: 5; 2024: 2.

10 papers.

  1. 2021How Transferable Are Self-supervised Features in Medical Image Classification Tasks?cited by 32
  2. 2023A Review of Predictive and Contrastive Self-supervised Learning for Medical Imagescited by 10
  3. 2023When Multi-Task Learning Meets Partial Supervision: A Computer Vision Reviewcited by 6
  4. 2024LoRKD: Low-Rank Knowledge Decomposition for Medical Foundation Modelscited by 4
  5. 2023Assessing the Generalizability of Deep Neural Networks-Based Models for Black Skin Lesionscited by 3
  6. 2021Improved skin lesion recognition by a Self-Supervised Curricular Deep Learning approachcited by 2
  7. 2022Simple Control Baselines for Evaluating Transfer Learningcited by 1
  8. 2023Self-Supervised Multi-Modality Learning for Multi-Label Skin Lesion Classificationcited by 1
  9. 2023Self-supervised learning for hotspot detection and isolation from thermal imagescited by 0
  10. 2024Are Explanations Helpful? A Comparative Analysis of Explainability Methods in Skin Lesion Classifierscited by 0