# Dense Self-Supervised Learning for Medical Image Segmentation

arXiv: https://arxiv.org/abs/2407.20395 · HTML: https://mlsanity.com/paper/2407.20395/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 35 references that are in the map.
- First posted on arXiv 29 Jul 2024.

## Builds on (most cited first, 28 of 28)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [Unsupervised Visual Representation Learning by Context Prediction](https://mlsanity.com/paper/1505.05192/) (2015)
- [Dense Contrastive Learning for Self-Supervised Visual Pre-Training](https://mlsanity.com/paper/2011.09157/) (2020)
- [Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis](https://mlsanity.com/paper/2111.14791/) (2021)
- [Uncertainty-aware Self-ensembling Model for Semi-supervised 3D Left Atrium Segmentation](https://mlsanity.com/paper/1907.07034/) (2019)
- [Contrastive learning of global and local features for medical image segmentation with limited annotations](https://mlsanity.com/paper/2006.10511/) (2020)
- [Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/2011.10043/) (2020)
- [Unsupervised Learning of Dense Visual Representations](https://mlsanity.com/paper/2011.05499/) (2020)
- [VICRegL: Self-Supervised Learning of Local Visual Features](https://mlsanity.com/paper/2210.01571/) (2022)
- [Pixel Contrastive-Consistent Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2108.09025/) (2021)
- [Contrastive Learning for Label-Efficient Semantic Segmentation](https://mlsanity.com/paper/2012.06985/) (2020)
- [SAM: Self-supervised Learning of Pixel-wise Anatomical Embeddings in Radiological Images](https://mlsanity.com/paper/2012.02383/) (2020)
- [Positional Contrastive Learning for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/2106.09157/) (2021)
- [Semi-supervised Contrastive Learning for Label-efficient Medical Image Segmentation](https://mlsanity.com/paper/2109.07407/) (2021)
- [Anatomical Priors in Convolutional Networks for Unsupervised Biomedical Segmentation](https://mlsanity.com/paper/1903.03148/) (2019)
- [Self-Supervised Pretraining for 2D Medical Image Segmentation](https://mlsanity.com/paper/2209.00314/) (2022)
- [vox2vec: A Framework for Self-supervised Contrastive Learning of Voxel-level Representations in Medical Images](https://mlsanity.com/paper/2307.14725/) (2023)
- [BT-Unet: A self-supervised learning framework for biomedical image segmentation using Barlow Twins with U-Net models](https://mlsanity.com/paper/2112.03916/) (2021)
- [S5CL: Unifying Fully-Supervised, Self-Supervised, and Semi-Supervised Learning Through Hierarchical Contrastive Learning](https://mlsanity.com/paper/2203.07307/) (2022)

## Built on by (most cited first, 3 of 3)

- [CoRe: Joint Optimization with Contrastive Learning for Medical Image Registration](https://mlsanity.com/paper/2603.23694/) (2026)
- [Evaluating self-supervised echocardiographic representations across downstream extraction strategies for left-ventricular segmentation and ejection fraction estimation](https://mlsanity.com/paper/2606.22943/) (2026)
- [Pix2Rep-v2: Data-Efficient Representation Learning for Dense Medical Imaging Applications](https://mlsanity.com/paper/2609.01427/) (2026)

## Abstract (arXiv)

Deep learning has revolutionized medical image segmentation, but it relies heavily on high-quality annotations. The time, cost and expertise required to label images at the pixel-level for each new task has slowed down widespread adoption of the paradigm. We propose Pix2Rep, a self-supervised learning (SSL) approach for few-shot segmentation, that reduces the manual annotation burden by learning powerful pixel-level representations directly from unlabeled images. Pix2Rep is a novel pixel-level loss and pre-training paradigm for contrastive SSL on whole images. It is applied to generic encoder-decoder deep learning backbones (e.g., U-Net). Whereas most SSL methods enforce invariance of the learned image-level representations under intensity and spatial image augmentations, Pix2Rep enforces equivariance of the pixel-level representations. We demonstrate the framework on a task of cardiac MRI segmentation. Results show improved performance compared to existing semi- and self-supervised approaches; and a 5-fold reduction in the annotation burden for equivalent performance versus a fully supervised U-Net baseline. This includes a 30% (resp. 31%) DICE improvement for one-shot segmentation under linear-probing (resp. fine-tuning). Finally, we also integrate the novel Pix2Rep concept with the Barlow Twins non-contrastive SSL, which leads to even better segmentation performance.