mlsanityData to 13 Oct 2026

WPS-SAM: Towards Weakly-Supervised Part Segmentation with Foundation Models

arXiv 2407.10131 ↗First posted 14 Jul 2024cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Segmenting and recognizing diverse object parts is crucial in computer vision and robotics. Despite significant progress in object segmentation, part-level segmentation remains underexplored due to complex boundaries and scarce annotated data. To address this, we propose a novel Weakly-supervised Part Segmentation (WPS) setting and an approach called WPS-SAM, built on the large-scale pre-trained vision foundation model, Segment Anything Model (SAM). WPS-SAM is an end-to-end framework designed to extract prompt tokens directly from images and perform pixel-level segmentation of part regions. During its training phase, it only uses weakly supervised labels in the form of bounding boxes or points. Extensive experiments demonstrate that, through exploiting the rich knowledge embedded in pre-trained foundation models, WPS-SAM outperforms other segmentation models trained with pixel-level strong annotations. Specifically, WPS-SAM achieves 68.93% mIOU and 79.53% mACC on the PartImageNet dataset, surpassing state-of-the-art fully supervised methods by approximately 4% in terms of mIOU.

Key facts. Cited by 4 papers in the map (data through 13 Oct 2026). Builds on 59 of its 78 references that are in the map. First posted on arXiv 14 Jul 2024.

Builds on

59 of 78 references.

References · 59

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,950
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,588
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,043
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,322
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,111
  7. 2015Distilling the Knowledge in a Neural Networkcited by 10,180
  8. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,625
  9. 2023Segment Anythingcited by 8,195
  10. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,857
  11. 2020End-to-End Object Detection with Transformerscited by 7,396
  12. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,364
38 more
  1. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,849
  2. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,855
  3. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,216
  4. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,930
  5. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,159
  6. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,150
  7. 2023Segment Everything Everywhere All at Oncecited by 547
  8. 2018Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentationcited by 321
  9. 2023Fast Segment Anythingcited by 287
  10. 2015BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentationcited by 238
  11. 2014Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Partscited by 231
  12. 2016Simple Does It: Weakly Supervised Instance and Semantic Segmentationcited by 226
  13. 2017Object Region Mining with Adversarial Erasing: A Simple Classification to Semantic Segmentation Approachcited by 201
  14. 2023SegGPT: Segmenting Everything In Contextcited by 193
  15. 2023EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anythingcited by 187
  16. 2023Semantic-SAM: Segment and Recognize Anything at Any Granularitycited by 185
  17. 2023RSPrompter: Learning to Prompt for Remote Sensing Instance Segmentation based on Visual Foundation Modelcited by 180
  18. 2020Single-Stage Semantic Segmentation from Image Labelscited by 167
  19. 2023PACO: Parts and Attributes of Common Objectscited by 146
  20. 2023Matcher: Segment Anything with One Shot Using All-Purpose Feature Matchingcited by 142
  21. 2015End-to-end people detection in crowded scenescited by 140
  22. 2021Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentationcited by 134
  23. 2021Railroad is not a Train: Saliency as Pseudo-pixel Supervision for Weakly Supervised Semantic Segmentationcited by 127
  24. 2018On Regularized Losses for Weakly-supervised CNN Segmentationcited by 121
  25. 2023Segment Anything Is Not Always Perfect: An Investigation of SAM on Different Real-world Applicationscited by 121
  26. 2022Learning Affinity from Attention: End-to-End Weakly-Supervised Semantic Segmentation with Transformerscited by 118
  27. 2022CLIP is Also an Efficient Segmenter: A Text-Driven Approach for Weakly Supervised Semantic Segmentationcited by 118
  28. 2021PartImageNet: A Large, High-Quality Dataset of Partscited by 107
  29. 2015Constrained Convolutional Neural Networks for Weakly Supervised Segmentationcited by 100
  30. 2019SCOPS: Self-Supervised Co-Part Segmentationcited by 100
  31. 2019Box-driven Class-wise Region Masking and Filling Rate Guided Loss for Weakly Supervised Semantic Segmentationcited by 97
  32. 2019Reliability Does Matter: An End-to-End Weakly Supervised Semantic Segmentation Approachcited by 88
  33. 2021Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrastcited by 85
  34. 2018CIAN: Cross-Image Affinity Net for Weakly Supervised Semantic Segmentationcited by 80
  35. 2017Unsupervised learning of object landmarks by factorized spatial embeddingscited by 64
  36. 2022Regional Semantic Contrast and Aggregation for Weakly Supervised Semantic Segmentationcited by 64
  37. 2023Segment Anything Model (SAM) Enhanced Pseudo Labels for Weakly Supervised Semantic Segmentationcited by 53
  38. 2023Foundation Model Assisted Weakly Supervised Semantic Segmentationcited by 32

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year

Citing papers per year: 2025: 4.

4 papers.

  1. 2025ProtoGCD: Unified and Unbiased Prototype Learning for Generalized Category Discoverycited by 19
  2. 2025Mitigating Spurious Correlations in Weakly Supervised Semantic Segmentation via Cross-architecture Consistency Regularizationcited by 1
  3. 2025Segment Any Vehicle: Semantic and Visual Context Driven SAM and A Benchmarkcited by 1
  4. 2025ReSAM: Refine, Requery, and Reinforce: Self-Prompting Point-Supervised Segmentation for Remote Sensing Imagescited by 1