# Contrastive Learning for Label-Efficient Semantic Segmentation

arXiv: https://arxiv.org/abs/2012.06985 · HTML: https://mlsanity.com/paper/2012.06985/

## Key facts

- Cited by 93 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Dec 2020.

## Built on by (most cited first, 30 of 93)

- [Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels](https://mlsanity.com/paper/2203.03884/) (2022)
- [Pixel Contrastive-Consistent Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2108.09025/) (2021)
- [Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast](https://mlsanity.com/paper/2110.07110/) (2021)
- [Bootstrapping Semantic Segmentation with Regional Contrast](https://mlsanity.com/paper/2104.04465/) (2021)
- [WildNet: Learning Domain Generalized Semantic Segmentation from the Wild](https://mlsanity.com/paper/2204.01446/) (2022)
- [Contrastive Boundary Learning for Point Cloud Segmentation](https://mlsanity.com/paper/2203.05272/) (2022)
- [Contrastive Learning for Neural Topic Model](https://mlsanity.com/paper/2110.12764/) (2021)
- [Local contrastive loss with pseudo-label based self-training for semi-supervised medical image segmentation](https://mlsanity.com/paper/2112.09645/) (2021)
- [Leveraging Hidden Positives for Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2303.15014/) (2023)
- [Mask Grounding for Referring Image Segmentation](https://mlsanity.com/paper/2312.12198/) (2023)
- [Mitigating Data Sparsity for Short Text Topic Modeling by Topic-Semantic Contrastive Learning](https://mlsanity.com/paper/2211.12878/) (2022)
- [Self-Paced Contrastive Learning for Semi-supervised Medical Image Segmentation with Meta-labels](https://mlsanity.com/paper/2107.13741/) (2021)
- [Exploring Intra- and Inter-Video Relation for Surgical Semantic Scene Segmentation](https://mlsanity.com/paper/2203.15251/) (2022)
- [CP2: Copy-Paste Contrastive Pretraining for Semantic Segmentation](https://mlsanity.com/paper/2203.11709/) (2022)
- [Local Spatiotemporal Representation Learning for Longitudinally-consistent Neuroimage Analysis](https://mlsanity.com/paper/2206.04281/) (2022)
- [Semi-supervised Crowd Counting via Density Agency](https://mlsanity.com/paper/2209.02955/) (2022)
- [Adaptive Contrast for Image Regression in Computer-Aided Disease Assessment](https://mlsanity.com/paper/2112.11700/) (2021)
- [Towards the Uncharted: Density-Descending Feature Perturbation for Semi-supervised Semantic Segmentation](https://mlsanity.com/paper/2403.06462/) (2024)
- [Modeling Dynamic Topics in Chain-Free Fashion by Evolution-Tracking Contrastive Learning and Unassociated Word Exclusion](https://mlsanity.com/paper/2405.17957/) (2024)
- [Exploring Multi-view Pixel Contrast for General and Robust Image Forgery Localization](https://mlsanity.com/paper/2406.13565/) (2024)
- [Bridge to Target Domain by Prototypical Contrastive Learning and Label Confusion: Re-explore Zero-Shot Learning for Slot Filling](https://mlsanity.com/paper/2110.03572/) (2021)
- [CLUDA : Contrastive Learning in Unsupervised Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2208.14227/) (2022)
- [Contrastive Learning for Automotive mmWave Radar Detection Points Based Instance Segmentation](https://mlsanity.com/paper/2203.06553/) (2022)
- [Exploring Feature Representation Learning for Semi-supervised Medical Image Segmentation](https://mlsanity.com/paper/2111.10989/) (2021)
- [Separated Contrastive Learning for Organ-at-Risk and Gross-Tumor-Volume Segmentation with Limited Annotation](https://mlsanity.com/paper/2112.02743/) (2021)
- [ConCL: Concept Contrastive Learning for Dense Prediction Pre-training in Pathology Images](https://mlsanity.com/paper/2207.06733/) (2022)
- [Semi-Supervised Semantic Segmentation with Cross Teacher Training](https://mlsanity.com/paper/2209.01327/) (2022)
- [Towards Galaxy Foundation Models with Hybrid Contrastive Learning](https://mlsanity.com/paper/2206.11927/) (2022)
- [Fine-Grained Off-Road Semantic Segmentation and Mapping via Contrastive Learning](https://mlsanity.com/paper/2103.03651/) (2021)
- [Learning Shadow Correspondence for Video Shadow Detection](https://mlsanity.com/paper/2208.00150/) (2022)

## Abstract (arXiv)

Collecting labeled data for the task of semantic segmentation is expensive and time-consuming, as it requires dense pixel-level annotations. While recent Convolutional Neural Network (CNN) based semantic segmentation approaches have achieved impressive results by using large amounts of labeled training data, their performance drops significantly as the amount of labeled data decreases. This happens because deep CNNs trained with the de facto cross-entropy loss can easily overfit to small amounts of labeled data. To address this issue, we propose a simple and effective contrastive learning-based training strategy in which we first pretrain the network using a pixel-wise, label-based contrastive loss, and then fine-tune it using the cross-entropy loss. This approach increases intra-class compactness and inter-class separability, thereby resulting in a better pixel classifier. We demonstrate the effectiveness of the proposed training strategy using the Cityscapes and PASCAL VOC 2012 segmentation datasets. Our results show that pretraining with the proposed contrastive loss results in large performance gains (more than 20% absolute improvement in some settings) when the amount of labeled data is limited. In many settings, the proposed contrastive pretraining strategy, which does not use any additional data, is able to match or outperform the widely-used ImageNet pretraining strategy that uses more than a million additional labeled images.