# Self-supervision for medical image classification: state-of-the-art performance with ~100 labeled training samples per class

arXiv: https://arxiv.org/abs/2304.05163 · HTML: https://mlsanity.com/paper/2304.05163/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 33 references that are in the map.
- First posted on arXiv 11 Apr 2023.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)](https://mlsanity.com/paper/1902.03368/) (2019)
- [Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/1902.06162/) (2019)
- [Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)](https://mlsanity.com/paper/1710.05006/) (2017)
- [Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1906.12340/) (2019)
- [XCiT: Cross-Covariance Image Transformers](https://mlsanity.com/paper/2106.09681/) (2021)
- [Big Self-Supervised Models Advance Medical Image Classification](https://mlsanity.com/paper/2101.05224/) (2021)
- [BCN20000: Dermoscopic Lesions in the Wild](https://mlsanity.com/paper/1908.02288/) (2019)
- [A Patient-Centric Dataset of Images and Metadata for Identifying Melanomas Using Clinical Context](https://mlsanity.com/paper/2008.07360/) (2020)
- [When Does Contrastive Visual Representation Learning Work?](https://mlsanity.com/paper/2105.05837/) (2021)
- [Skin Lesion Classification Using Ensembles of Multi-Resolution EfficientNets with Meta Data](https://mlsanity.com/paper/1910.03910/) (2019)
- [A Study of the Generalizability of Self-Supervised Representations](https://mlsanity.com/paper/2109.09150/) (2021)

## Built on by (most cited first, 7 of 7)

- [Medical Slice Transformer: Improved Diagnosis and Explainability on 3D Medical Images with DINOv2](https://mlsanity.com/paper/2411.15802/) (2024)
- [Graph Neural Networks: A suitable Alternative to MLPs in Latent 3D Medical Image Classification?](https://mlsanity.com/paper/2407.17219/) (2024)
- [Medical supervised masked autoencoders: Crafting a better masking strategy and efficient fine-tuning schedule for medical image classification](https://mlsanity.com/paper/2305.05871/) (2023)
- [DINO-LG: Enhancing Vision Transformers with Label Guidance for Coronary Artery Calcium Detection](https://mlsanity.com/paper/2411.07976/) (2024)
- [Cluster-based human-in-the-loop strategy for improving machine learning-based circulating tumor cell detection in liquid biopsy](https://mlsanity.com/paper/2411.16332/) (2024)
- [Entropy-Guided Self-Supervised Learning for Medical Image Classification](https://mlsanity.com/paper/2605.21970/) (2026)
- [Self-Supervised Visual Representation Learning: Pretrain-Finetuning or Joint Training?](https://mlsanity.com/paper/2607.13192/) (2026)

## Abstract (arXiv)

Is self-supervised deep learning (DL) for medical image analysis already a serious alternative to the de facto standard of end-to-end trained supervised DL? We tackle this question for medical image classification, with a particular focus on one of the currently most limiting factors of the field: the (non-)availability of labeled data. Based on three common medical imaging modalities (bone marrow microscopy, gastrointestinal endoscopy, dermoscopy) and publicly available data sets, we analyze the performance of self-supervised DL within the self-distillation with no labels (DINO) framework. After learning an image representation without use of image labels, conventional machine learning classifiers are applied. The classifiers are fit using a systematically varied number of labeled data (1-1000 samples per class). Exploiting the learned image representation, we achieve state-of-the-art classification performance for all three imaging modalities and data sets with only a fraction of between 1% and 10% of the available labeled data and about 100 labeled samples per class.