# Segment Anything is A Good Pseudo-label Generator for Weakly Supervised Semantic Segmentation

arXiv: https://arxiv.org/abs/2305.01275 · HTML: https://mlsanity.com/paper/2305.01275/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 46 references that are in the map.
- First posted on arXiv 2 May 2023.

## Builds on (most cited first, 30 of 37)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [EVA: Exploring the Limits of Masked Visual Representation Learning at Scale](https://mlsanity.com/paper/2211.07636/) (2022)
- [Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks](https://mlsanity.com/paper/1910.01279/) (2019)
- [ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1604.05144/) (2016)
- [Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/1803.10464/) (2018)
- [Self-supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2004.04581/) (2020)
- [What's the Point: Semantic Segmentation with Point Supervision](https://mlsanity.com/paper/1506.02106/) (2015)
- [Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relations](https://mlsanity.com/paper/1904.05044/) (2019)
- [BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1503.01640/) (2015)
- [Simple Does It: Weakly Supervised Instance and Semantic Segmentation](https://mlsanity.com/paper/1603.07485/) (2016)
- [A Simple Pooling-Based Design for Real-Time Salient Object Detection](https://mlsanity.com/paper/1904.09569/) (2019)
- [Object Region Mining with Adversarial Erasing: A Simple Classification to Semantic Segmentation Approach](https://mlsanity.com/paper/1703.08448/) (2017)
- [Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation](https://mlsanity.com/paper/1502.02734/) (2015)
- [Wider or Deeper: Revisiting the ResNet Model for Visual Recognition](https://mlsanity.com/paper/1611.10080/) (2016)
- [Revisiting Dilated Convolution: A Simple Approach for Weakly- and Semi- Supervised Semantic Segmentation](https://mlsanity.com/paper/1805.04574/) (2018)
- [Mining Cross-Image Semantics for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2007.01947/) (2020)
- [Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2103.08896/) (2021)
- [Railroad is not a Train: Saliency as Pseudo-pixel Supervision for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2105.08965/) (2021)
- [CLIP is Also an Efficient Segmenter: A Text-Driven Approach for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2212.09506/) (2022)

## Built on by (most cited first, 27 of 27)

- [A Comprehensive Survey on Segment Anything Model for Vision and Beyond](https://mlsanity.com/paper/2305.08196/) (2023)
- [Annotation-free Audio-Visual Segmentation](https://mlsanity.com/paper/2305.11019/) (2023)
- [A Survey on Segment Anything Model (SAM): Vision Foundation Model Meets Prompt Engineering](https://mlsanity.com/paper/2306.06211/) (2023)
- [MedCLIP-SAMv2: Towards Universal Text-Driven Medical Image Segmentation](https://mlsanity.com/paper/2409.19483/) (2024)
- [SAM-Assisted Remote Sensing Imagery Semantic Segmentation with Object and Boundary Constraints](https://mlsanity.com/paper/2312.02464/) (2023)
- [Foundation Model Assisted Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2312.03585/) (2023)
- [Semantic-aware SAM for Point-Prompted Instance Segmentation](https://mlsanity.com/paper/2312.15895/) (2023)
- [Towards Training-free Open-world Segmentation via Image Prompt Foundation Models](https://mlsanity.com/paper/2310.10912/) (2023)
- [WeakSAM: Segment Anything Meets Weakly-supervised Instance-level Recognition](https://mlsanity.com/paper/2402.14812/) (2024)
- [iSeg: An Iterative Refinement-based Framework for Training-free Segmentation](https://mlsanity.com/paper/2409.03209/) (2024)
- [Weakly-Supervised Affordance Grounding Guided by Part-Level Semantic Priors](https://mlsanity.com/paper/2505.24103/) (2025)
- [There is no SAMantics! Exploring SAM as a Backbone for Visual Understanding Tasks](https://mlsanity.com/paper/2411.15288/) (2024)
- [Tuning-free Universally-Supervised Semantic Segmentation](https://mlsanity.com/paper/2405.14294/) (2024)
- [Industrial Segment Anything -- a Case Study in Aircraft Manufacturing, Intralogistics, Maintenance, Repair, and Overhaul](https://mlsanity.com/paper/2307.12674/) (2023)
- [Semantic Prompt Learning for Weakly-Supervised Semantic Segmentation](https://mlsanity.com/paper/2401.11791/) (2024)
- [DHR: Dual Features-Driven Hierarchical Rebalancing in Inter- and Intra-Class Regions for Weakly-Supervised Semantic Segmentation](https://mlsanity.com/paper/2404.00380/) (2024)
- [Segment Anything Model for automated image data annotation: empirical studies using text prompts from Grounding DINO](https://mlsanity.com/paper/2406.19057/) (2024)
- [Boosting Weakly-Supervised Image Segmentation via Representation, Transform, and Compensator](https://mlsanity.com/paper/2309.00871/) (2023)
- [Segment Anything Model-guided Collaborative Learning Network for Scribble-supervised Polyp Segmentation](https://mlsanity.com/paper/2312.00312/) (2023)
- [Mixed-Query Transformer: A Unified Image Segmentation Architecture](https://mlsanity.com/paper/2404.04469/) (2024)
- [FusionVision: A comprehensive approach of 3D object reconstruction and segmentation from RGB-D cameras using YOLO and fast segment anything](https://mlsanity.com/paper/2403.00175/) (2024)
- [Weakly Supervised Segmentation as Semantic-Based Regularization](https://mlsanity.com/paper/2605.13674/) (2026)
- [Enhancing Weakly Supervised Semantic Segmentation with Multi-modal Foundation Models: An End-to-End Approach](https://mlsanity.com/paper/2405.06586/) (2024)
- [SinkSAM-Net: Knowledge-Driven Self-Supervised Sinkhole Segmentation Using Topographic Priors and Segment Anything Model](https://mlsanity.com/paper/2410.01473/) (2024)
- [Depth Edge Alignment Loss: DEALing with Depth in Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2509.17702/) (2025)
- [SAPNet++: Evolving Point-Prompted Instance Segmentation with Semantic and Spatial Awareness](https://mlsanity.com/paper/2602.21762/) (2026)
- [Exploring SAM Supervision for Fine-Grained UAV Target Segmentation under Data Scarcity](https://mlsanity.com/paper/2607.03754/) (2026)

## Abstract (arXiv)

Weakly supervised semantic segmentation with weak labels is a long-lived ill-posed problem. Mainstream methods mainly focus on improving the quality of pseudo labels. In this report, we attempt to explore the potential of 'prompt to masks' from the powerful class-agnostic large segmentation model, segment-anything. Specifically, different weak labels are used as prompts to the segment-anything model, generating precise class masks. The class masks are utilized to generate pseudo labels to train the segmentation networks. We have conducted extensive experiments on PASCAL VOC 2012 dataset. Experiments demonstrate that segment-anything can serve as a good pseudo-label generator. The code will be made publicly available.