# Self-Supervised Pretraining for 2D Medical Image Segmentation

arXiv: https://arxiv.org/abs/2209.00314 · HTML: https://mlsanity.com/paper/2209.00314/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 28 references that are in the map.
- First posted on arXiv 1 Sep 2022.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [Unsupervised Visual Representation Learning by Context Prediction](https://mlsanity.com/paper/1505.05192/) (2015)
- [Data-Efficient Image Recognition with Contrastive Predictive Coding](https://mlsanity.com/paper/1905.09272/) (2019)
- [Deep Learning Scaling is Predictable, Empirically](https://mlsanity.com/paper/1712.00409/) (2017)
- [Big Self-Supervised Models Advance Medical Image Classification](https://mlsanity.com/paper/2101.05224/) (2021)
- [Self-supervised Pretraining of Visual Features in the Wild](https://mlsanity.com/paper/2103.01988/) (2021)
- [Self supervised contrastive learning for digital histopathology](https://mlsanity.com/paper/2011.13971/) (2020)
- [Solo-learn: A Library of Self-supervised Methods for Visual Representation Learning](https://mlsanity.com/paper/2108.01775/) (2021)
- [Self-Supervised Pretraining Improves Self-Supervised Pretraining](https://mlsanity.com/paper/2103.12718/) (2021)
- [A scoping review of transfer learning research on medical image analysis using ImageNet](https://mlsanity.com/paper/2004.13175/) (2020)

## Built on by (most cited first, 24 of 24)

- [Towards Geospatial Foundation Models via Continual Pretraining](https://mlsanity.com/paper/2302.04476/) (2023)
- [Continual Self-supervised Learning: Towards Universal Multi-modal Medical Data Representation Learning](https://mlsanity.com/paper/2311.17597/) (2023)
- [Review of multimodal machine learning approaches in healthcare](https://mlsanity.com/paper/2402.02460/) (2024)
- [Multi-Label Guided Soft Contrastive Learning for Efficient Earth Observation Pretraining](https://mlsanity.com/paper/2405.20462/) (2024)
- [From Denoising Training to Test-Time Adaptation: Enhancing Domain Generalization for Medical Image Segmentation](https://mlsanity.com/paper/2310.20271/) (2023)
- [Disruptive Autoencoders: Leveraging Low-level features for 3D Medical Image Pre-training](https://mlsanity.com/paper/2307.16896/) (2023)
- [Can pre-trained models assist in dataset distillation?](https://mlsanity.com/paper/2310.03295/) (2023)
- [Self-supervised learning via inter-modal reconstruction and feature projection networks for label-efficient 3D-to-2D segmentation](https://mlsanity.com/paper/2307.03008/) (2023)
- [Self-supervised Semantic Segmentation: Consistency over Transformation](https://mlsanity.com/paper/2309.00143/) (2023)
- [MLVICX: Multi-Level Variance-Covariance Exploration for Chest X-ray Self-Supervised Representation Learning](https://mlsanity.com/paper/2403.11504/) (2024)
- [Cross-sensor self-supervised training and alignment for remote sensing](https://mlsanity.com/paper/2405.09922/) (2024)
- [CycleSAM: Few-Shot Surgical Scene Segmentation with Cycle- and Scene-Consistent Feature Matching](https://mlsanity.com/paper/2407.06795/) (2024)
- [Dense Self-Supervised Learning for Medical Image Segmentation](https://mlsanity.com/paper/2407.20395/) (2024)
- [Exploring Foundation Models in Remote Sensing Image Change Detection: A Comprehensive Survey](https://mlsanity.com/paper/2410.07824/) (2024)
- [Self-Supervised Learning for Image Segmentation: A Comprehensive Survey](https://mlsanity.com/paper/2505.13584/) (2025)
- [A Study on Self-Supervised Pretraining for Vision Problems in Gastrointestinal Endoscopy](https://mlsanity.com/paper/2401.06278/) (2024)
- [Explanations of Classifiers Enhance Medical Image Segmentation via End-to-end Pre-training](https://mlsanity.com/paper/2401.08469/) (2024)
- [Exploring Intrinsic Properties of Medical Images for Self-Supervised Binary Semantic Segmentation](https://mlsanity.com/paper/2402.02367/) (2024)
- [Self-supervised Pretraining for Cardiovascular Magnetic Resonance Cine Segmentation](https://mlsanity.com/paper/2409.18100/) (2024)
- [Enhancing pretraining efficiency for medical image segmentation via transferability metrics](https://mlsanity.com/paper/2410.18677/) (2024)
- [MatSSL: Robust Self-Supervised Representation Learning for Metallographic Image Segmentation](https://mlsanity.com/paper/2507.18184/) (2025)
- [Unsupervised Segmentation of Micro-CT Scans of Polyurethane Structures By Combining Hidden-Markov-Random Fields and a U-Net](https://mlsanity.com/paper/2511.11378/) (2025)
- [Benchmarking transferability of SSL pretraining to same and different modality segmentation tasks](https://mlsanity.com/paper/2605.18491/) (2026)
- [Self-Supervised Vision Transformers for CBCT-Based Detection of Temporomandibular Joint Osteoarthritis](https://mlsanity.com/paper/2606.08364/) (2026)

## Abstract (arXiv)

Supervised machine learning provides state-of-the-art solutions to a wide range of computer vision problems. However, the need for copious labelled training data limits the capabilities of these algorithms in scenarios where such input is scarce or expensive. Self-supervised learning offers a way to lower the need for manually annotated data by pretraining models for a specific domain on unlabelled data. In this approach, labelled data are solely required to fine-tune models for downstream tasks. Medical image segmentation is a field where labelling data requires expert knowledge and collecting large labelled datasets is challenging; therefore, self-supervised learning algorithms promise substantial improvements in this field. Despite this, self-supervised learning algorithms are used rarely to pretrain medical image segmentation networks. In this paper, we elaborate and analyse the effectiveness of supervised and self-supervised pretraining approaches on downstream medical image segmentation, focusing on convergence and data efficiency. We find that self-supervised pretraining on natural images and target-domain-specific images leads to the fastest and most stable downstream convergence. In our experiments on the ACDC cardiac segmentation dataset, this pretraining approach achieves 4-5 times faster fine-tuning convergence compared to an ImageNet pretrained model. We also show that this approach requires less than five epochs of pretraining on domain-specific data to achieve such improvement in the downstream convergence time. Finally, we find that, in low-data scenarios, supervised ImageNet pretraining achieves the best accuracy, requiring less than 100 annotated samples to realise close to minimal error.