mlsanityData to 7 Oct 2026

Dense Self-Supervised Learning for Medical Image Segmentation

arXiv 2407.20395 ↗First posted 29 Jul 2024cs.CV cs.AI cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Deep learning has revolutionized medical image segmentation, but it relies heavily on high-quality annotations. The time, cost and expertise required to label images at the pixel-level for each new task has slowed down widespread adoption of the paradigm. We propose Pix2Rep, a self-supervised learning (SSL) approach for few-shot segmentation, that reduces the manual annotation burden by learning powerful pixel-level representations directly from unlabeled images. Pix2Rep is a novel pixel-level loss and pre-training paradigm for contrastive SSL on whole images. It is applied to generic encoder-decoder deep learning backbones (e.g., U-Net). Whereas most SSL methods enforce invariance of the learned image-level representations under intensity and spatial image augmentations, Pix2Rep enforces equivariance of the pixel-level representations. We demonstrate the framework on a task of cardiac MRI segmentation. Results show improved performance compared to existing semi- and self-supervised approaches; and a 5-fold reduction in the annotation burden for equivalent performance versus a fully supervised U-Net baseline. This includes a 30% (resp. 31%) DICE improvement for one-shot segmentation under linear-probing (resp. fine-tuning). Finally, we also integrate the novel Pix2Rep concept with the Barlow Twins non-contrastive SSL, which leads to even better segmentation performance.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 28 of its 35 references that are in the map. First posted on arXiv 29 Jul 2024.

Builds on

28 of 35 references.

References · 28

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  6. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
  7. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  8. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  9. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  10. 2016Colorful Image Colorizationcited by 1,113
  11. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  12. 2020Dense Contrastive Learning for Self-Supervised Visual Pre-Trainingcited by 512
16 more
  1. 2021Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysiscited by 418
  2. 2019Uncertainty-aware Self-ensembling Model for Semi-supervised 3D Left Atrium Segmentationcited by 378
  3. 2020Contrastive learning of global and local features for medical image segmentation with limited annotationscited by 309
  4. 2020Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learningcited by 297
  5. 2020Unsupervised Learning of Dense Visual Representationscited by 147
  6. 2022VICRegL: Self-Supervised Learning of Local Visual Featurescited by 118
  7. 2021Pixel Contrastive-Consistent Semi-Supervised Semantic Segmentationcited by 101
  8. 2020Contrastive Learning for Label-Efficient Semantic Segmentationcited by 93
  9. 2020SAM: Self-supervised Learning of Pixel-wise Anatomical Embeddings in Radiological Imagescited by 66
  10. 2021Positional Contrastive Learning for Volumetric Medical Image Segmentationcited by 56
  11. 2021Semi-supervised Contrastive Learning for Label-efficient Medical Image Segmentationcited by 52
  12. 2019Anatomical Priors in Convolutional Networks for Unsupervised Biomedical Segmentationcited by 40
  13. 2022Self-Supervised Pretraining for 2D Medical Image Segmentationcited by 24
  14. 2023vox2vec: A Framework for Self-supervised Contrastive Learning of Voxel-level Representations in Medical Imagescited by 19
  15. 2021BT-Unet: A self-supervised learning framework for biomedical image segmentation using Barlow Twins with U-Net modelscited by 12
  16. 2022S5CL: Unifying Fully-Supervised, Self-Supervised, and Semi-Supervised Learning Through Hierarchical Contrastive Learningcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026CoRe: Joint Optimization with Contrastive Learning for Medical Image Registrationcited by 0
  2. 2026Evaluating self-supervised echocardiographic representations across downstream extraction strategies for left-ventricular segmentation and ejection fraction estimationcited by 0
  3. 2026Pix2Rep-v2: Data-Efficient Representation Learning for Dense Medical Imaging Applicationscited by 0