mlsanityData to 7 Oct 2026

An Alternative to WSSS? An Empirical Study of the Segment Anything Model (SAM) on Weakly-Supervised Semantic Segmentation Problems

arXiv 2305.01586 ↗First posted 2 May 2023cs.CVcited by 19

Not on the map. Why?

Abstract (arXiv)

The Segment Anything Model (SAM) has demonstrated exceptional performance and versatility, making it a promising tool for various related tasks. In this report, we explore the application of SAM in Weakly-Supervised Semantic Segmentation (WSSS). Particularly, we adapt SAM as the pseudo-label generation pipeline given only the image-level class labels. While we observed impressive results in most cases, we also identify certain limitations. Our study includes performance evaluations on PASCAL VOC and MS-COCO, where we achieved remarkable improvements over the latest state-of-the-art methods on both datasets. We anticipate that this report encourages further explorations of adopting SAM in WSSS, as well as wider real-world applications.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 44 of its 54 references that are in the map. First posted on arXiv 2 May 2023.

Builds on

44 of 54 references.

References · 44

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  4. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  5. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  6. 2016ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentationcited by 330
  7. 2018Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentationcited by 321
  8. 2020Self-supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentationcited by 299
  9. 2015What's the Point: Semantic Segmentation with Point Supervisioncited by 297
  10. 2019Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relationscited by 245
  11. 2015BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentationcited by 238
  12. 2020Causal Intervention for Weakly-Supervised Semantic Segmentationcited by 229
32 more
  1. 2016Simple Does It: Weakly Supervised Instance and Semantic Segmentationcited by 226
  2. 2017Object Region Mining with Adversarial Erasing: A Simple Classification to Semantic Segmentation Approachcited by 201
  3. 2022Multi-class Token Transformer for Weakly Supervised Semantic Segmentationcited by 138
  4. 2021Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentationcited by 134
  5. 2021Railroad is not a Train: Saliency as Pseudo-pixel Supervision for Weakly Supervised Semantic Segmentationcited by 127
  6. 2018On Regularized Losses for Weakly-supervised CNN Segmentationcited by 121
  7. 2022Learning Affinity from Attention: End-to-End Weakly-Supervised Semantic Segmentation with Transformerscited by 118
  8. 2022CLIP is Also an Efficient Segmenter: A Text-Driven Approach for Weakly Supervised Semantic Segmentationcited by 118
  9. 2021BBAM: Bounding Box Attribution Map for Weakly Supervised Semantic and Instance Segmentationcited by 104
  10. 2018Learning random-walk label propagation for weakly-supervised semantic segmentationcited by 101
  11. 2019Box-driven Class-wise Region Masking and Filling Rate Guided Loss for Weakly Supervised Semantic Segmentationcited by 97
  12. 2021Non-Salient Region Object Mining for Weakly Supervised Semantic Segmentationcited by 95
  13. 2022Self-supervised Image-specific Prototype Exploration for Weakly Supervised Semantic Segmentationcited by 87
  14. 2021Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrastcited by 85
  15. 2018Weakly-Supervised Semantic Segmentation by Iteratively Mining Common Object Featurescited by 81
  16. 2021Leveraging Auxiliary Tasks with Affinity Learning for Weakly Supervised Semantic Segmentationcited by 77
  17. 2021Complementary Patch for Weakly Supervised Semantic Segmentationcited by 75
  18. 2022Class Re-Activation Maps for Weakly-Supervised Semantic Segmentationcited by 75
  19. 2021Context Decoupling Augmentation for Weakly Supervised Semantic Segmentationcited by 69
  20. 2021Reducing Information Bottleneck for Weakly Supervised Semantic Segmentationcited by 65
  21. 2022Weakly Supervised Semantic Segmentation using Out-of-Distribution Datacited by 58
  22. 2022L2G: A Simple Local-to-Global Knowledge Transfer Framework for Weakly Supervised Semantic Segmentationcited by 57
  23. 2021Discriminative Region Suppression for Weakly-Supervised Semantic Segmentationcited by 50
  24. 2021Pseudo-mask Matters in Weakly-supervised Semantic Segmentationcited by 48
  25. 2014Fully Convolutional Multi-Class Multiple Instance Learningcited by 37
  26. 2022Expansion and Shrinkage of Localization for Weakly-Supervised Semantic Segmentationcited by 37
  27. 2021Uncertainty Estimation via Response Scaling for Pseudo-mask Noise Mitigation in Weakly-supervised Semantic Segmentationcited by 28
  28. 2022Out-of-Candidate Rectification for Weakly Supervised Semantic Segmentationcited by 26
  29. 2022Weakly-Supervised Semantic Segmentation with Visual Words Learning and Hybrid Poolingcited by 19
  30. 20203D Guided Weakly Supervised Semantic Segmentationcited by 10
  31. 2021Inferring the Class Conditional Response Map for Weakly Supervised Semantic Segmentationcited by 9
  32. 2021GETAM: Gradient-weighted Element-wise Transformer Attention Map for Weakly-supervised Semantic segmentationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 7; 2025: 2; 2026: 2.

19 papers.

Sort:

By area:

  1. 2023A Comprehensive Survey on Segment Anything Model for Vision and Beyondcited by 64
  2. 2023A Survey on Segment Anything Model (SAM): Vision Foundation Model Meets Prompt Engineeringcited by 45
  3. 2023Foundation Model Assisted Weakly Supervised Semantic Segmentationcited by 31
  4. 2024WeakSAM: Segment Anything Meets Weakly-supervised Instance-level Recognitioncited by 9
  5. 2025Weakly-Supervised Affordance Grounding Guided by Part-Level Semantic Priorscited by 9
  6. 2024Tuning-free Universally-Supervised Semantic Segmentationcited by 6
  7. 2024Detecting Wildfires on UAVs with Real-time Segmentation Trained by Larger Teacher Modelscited by 5
  8. 2023Industrial Segment Anything -- a Case Study in Aircraft Manufacturing, Intralogistics, Maintenance, Repair, and Overhaulcited by 4
  9. 2023Exploring Limits of Diffusion-Synthetic Training with Weakly Supervised Semantic Segmentationcited by 4
  10. 2024WPS-SAM: Towards Weakly-Supervised Part Segmentation with Foundation Modelscited by 4
  11. 2023All-pairs Consistency Learning for Weakly Supervised Semantic Segmentationcited by 3
  12. 2024Semantic Prompt Learning for Weakly-Supervised Semantic Segmentationcited by 3
7 more
  1. 2024DHR: Dual Features-Driven Hierarchical Rebalancing in Inter- and Intra-Class Regions for Weakly-Supervised Semantic Segmentationcited by 3
  2. 2023Boosting Weakly-Supervised Image Segmentation via Representation, Transform, and Compensatorcited by 2
  3. 2023Segment Anything Model-guided Collaborative Learning Network for Scribble-supervised Polyp Segmentationcited by 2
  4. 2026Weakly Supervised Segmentation as Semantic-Based Regularizationcited by 1
  5. 2024Top-GAP: Integrating Size Priors in CNNs for more Interpretability, Robustness, and Bias Mitigationcited by 0
  6. 2025Depth Edge Alignment Loss: DEALing with Depth in Weakly Supervised Semantic Segmentationcited by 0
  7. 2026CRF Loss is How Networks Should Learn Boundaries in Weakly Supervised Segmentationcited by 0