# A Survey on Label-efficient Deep Image Segmentation: Bridging the Gap between Weak Supervision and Dense Prediction

arXiv: https://arxiv.org/abs/2207.01223 · HTML: https://mlsanity.com/paper/2207.01223/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 144 of its 184 references that are in the map.
- First posted on arXiv 4 Jul 2022.

## Builds on (most cited first, 30 of 144)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)

## Built on by (most cited first, 30 of 46)

- [Segment Anything in 3D with Radiance Fields](https://mlsanity.com/paper/2304.12308/) (2023)
- [Towards Label-free Scene Understanding by Vision Foundation Models](https://mlsanity.com/paper/2306.03899/) (2023)
- [ACSeg: Adaptive Conceptualization for Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2210.05944/) (2022)
- [Image Segmentation in Foundation Model Era: A Survey](https://mlsanity.com/paper/2408.12957/) (2024)
- [Point2Mask: Point-supervised Panoptic Segmentation via Optimal Transport](https://mlsanity.com/paper/2308.01779/) (2023)
- [WeakTr: Exploring Plain Vision Transformer for Weakly-supervised Semantic Segmentation](https://mlsanity.com/paper/2304.01184/) (2023)
- [A Survey of Label-Efficient Deep Learning for 3D Point Clouds](https://mlsanity.com/paper/2305.19812/) (2023)
- [Weakly-Supervised Semantic Segmentation with Image-Level Labels: from Traditional Models to Foundation Models](https://mlsanity.com/paper/2310.13026/) (2023)
- [A Token-level Text Image Foundation Model for Document Understanding](https://mlsanity.com/paper/2503.02304/) (2025)
- [AIO2: Online Correction of Object Labels for Deep Learning with Incomplete Annotation in Remote Sensing Image Segmentation](https://mlsanity.com/paper/2403.01641/) (2024)
- [Uncovering the Inner Workings of STEGO for Safe Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2304.07314/) (2023)
- [Label-efficient Segmentation via Affinity Propagation](https://mlsanity.com/paper/2310.10533/) (2023)
- [A Good Foundation is Worth Many Labels: Label-Efficient Panoptic Segmentation](https://mlsanity.com/paper/2405.19035/) (2024)
- [Decouple before Align: Visual Disentanglement Enhances Prompt Tuning](https://mlsanity.com/paper/2508.00395/) (2025)
- [Data efficient deep learning for medical image analysis: A survey](https://mlsanity.com/paper/2310.06557/) (2023)
- [LiDAR Remote Sensing Meets Weak Supervision: Concepts, Methods, and Perspectives](https://mlsanity.com/paper/2503.18384/) (2025)
- [SemiDAViL: Semi-supervised Domain Adaptation with Vision-Language Guidance for Semantic Segmentation](https://mlsanity.com/paper/2504.06389/) (2025)
- [UM-CAM: Uncertainty-weighted Multi-resolution Class Activation Maps for Weakly-supervised Fetal Brain Segmentation](https://mlsanity.com/paper/2306.11490/) (2023)
- [Test-time augmentation-based active learning and self-training for label-efficient segmentation](https://mlsanity.com/paper/2308.10727/) (2023)
- [A Literature Review of Literature Reviews in Pattern Analysis and Machine Intelligence](https://mlsanity.com/paper/2402.12928/) (2024)
- [From Few to More: Scribble-based Medical Image Segmentation via Masked Context Modeling and Continuous Pseudo Labels](https://mlsanity.com/paper/2408.12814/) (2024)
- [Exploring Generalizable Distillation for Efficient Medical Image Segmentation](https://mlsanity.com/paper/2207.12995/) (2022)
- [Partial Vessels Annotation-based Coronary Artery Segmentation with Self-training and Prototype Learning](https://mlsanity.com/paper/2307.04472/) (2023)
- [COIN: Counterfactual inpainting for weakly supervised semantic segmentation for medical images](https://mlsanity.com/paper/2404.12832/) (2024)
- [Point-supervised Brain Tumor Segmentation with Box-prompted MedSAM](https://mlsanity.com/paper/2408.00706/) (2024)
- [BroadCAM: Outcome-agnostic Class Activation Mapping for Small-scale Weakly Supervised Applications](https://mlsanity.com/paper/2309.03509/) (2023)
- [Generalized Robot 3D Vision-Language Model with Fast Rendering and Pre-Training Vision-Language Alignment](https://mlsanity.com/paper/2312.00663/) (2023)
- [Label-efficient multi-organ segmentation with a diffusion model](https://mlsanity.com/paper/2402.15216/) (2024)
- [Leveraging Fixed and Dynamic Pseudo-labels for Semi-supervised Medical Image Segmentation](https://mlsanity.com/paper/2405.07256/) (2024)
- [Weakly Supervised Lymph Nodes Segmentation Based on Partial Instance Annotations with Pre-trained Dual-branch Network and Pseudo Label Learning](https://mlsanity.com/paper/2408.09411/) (2024)

## Abstract (arXiv)

The rapid development of deep learning has made a great progress in image segmentation, one of the fundamental tasks of computer vision. However, the current segmentation algorithms mostly rely on the availability of pixel-level annotations, which are often expensive, tedious, and laborious. To alleviate this burden, the past years have witnessed an increasing attention in building label-efficient, deep-learning-based image segmentation algorithms. This paper offers a comprehensive review on label-efficient image segmentation methods. To this end, we first develop a taxonomy to organize these methods according to the supervision provided by different types of weak labels (including no supervision, inexact supervision, incomplete supervision and inaccurate supervision) and supplemented by the types of segmentation problems (including semantic segmentation, instance segmentation and panoptic segmentation). Next, we summarize the existing label-efficient image segmentation methods from a unified perspective that discusses an important question: how to bridge the gap between weak supervision and dense prediction -- the current methods are mostly based on heuristic priors, such as cross-pixel similarity, cross-label constraint, cross-view consistency, and cross-image relation. Finally, we share our opinions about the future research directions for label-efficient deep image segmentation.