mlsanityData to 7 Oct 2026

Visual and Textual Prior Guided Mask Assemble for Few-Shot Segmentation and Beyond

arXiv 2308.07539 ↗First posted 15 Aug 2023cs.CVcited by 19

Not on the map. Why?

Abstract (arXiv)

Few-shot segmentation (FSS) aims to segment the novel classes with a few annotated images. Due to CLIP's advantages of aligning visual and textual information, the integration of CLIP can enhance the generalization ability of FSS model. However, even with the CLIP model, the existing CLIP-based FSS methods are still subject to the biased prediction towards base classes, which is caused by the class-specific feature level interactions. To solve this issue, we propose a visual and textual Prior Guided Mask Assemble Network (PGMA-Net). It employs a class-agnostic mask assembly process to alleviate the bias, and formulates diverse tasks into a unified manner by assembling the prior through affinity. Specifically, the class-relevant textual and visual features are first transformed to class-agnostic prior in the form of probability map. Then, a Prior-Guided Mask Assemble Module (PGMAM) including multiple General Assemble Units (GAUs) is introduced. It considers diverse and plug-and-play interactions, such as visual-textual, inter- and intra-image, training-free, and high-order ones. Lastly, to ensure the class-agnostic ability, a Hierarchical Decoder with Channel-Drop Mechanism (HDCDM) is proposed to flexibly exploit the assembled masks and low-level features, without relying on any class-specific information. It achieves new state-of-the-art results in the FSS task, with mIoU of 77.6 on PASCAL-5i and 59.4 on COCO-20i in 1-shot scenario. Beyond this, we show that without extra re-training, the proposed PGMA-Net can solve bbox-level and cross-domain FSS, co-segmentation, zero-shot segmentation (ZSS) tasks, leading an any-shot segmentation framework.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 53 of its 83 references that are in the map. First posted on arXiv 15 Aug 2023.

Builds on

53 of 83 references.

References · 53

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  5. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  6. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  7. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  8. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  9. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  10. 2016Matching Networks for One Shot Learningcited by 2,246
  11. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
  12. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,139
38 more
  1. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
  2. 2022Language-driven Semantic Segmentationcited by 711
  3. 2021Extract Free Dense Labels from CLIPcited by 580
  4. 2019Meta-Learning with Implicit Gradientscited by 523
  5. 2021Image Segmentation Using Text and Image Promptscited by 509
  6. 2019PANet: Few-Shot Image Semantic Segmentation with Prototype Alignmentcited by 449
  7. 2022Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classificationcited by 355
  8. 2017One-Shot Learning for Semantic Segmentationcited by 337
  9. 2020Prior Guided Feature Enrichment Network for Few-Shot Segmentationcited by 271
  10. 2019Zero-Shot Semantic Segmentationcited by 245
  11. 2019CANet: Class-Agnostic Segmentation Networks with Iterative Refinement and Attentive Few-Shot Learningcited by 217
  12. 2021Adaptive Prototype Learning and Allocation for Few-Shot Segmentationcited by 207
  13. 2021Hypercorrelation Squeeze for Few-Shot Segmentationcited by 193
  14. 2020Prototype Mixture Models for Few-shot Semantic Segmentationcited by 190
  15. 2019Feature Weighting and Boosting for Few-Shot Segmentationcited by 189
  16. 2020Part-aware Prototype Network for Few-shot Semantic Segmentationcited by 162
  17. 2020Few-Shot Segmentation Without Meta-Learning: A Good Transductive Inference Is All You Need?cited by 135
  18. 2021End-to-End Referring Video Object Segmentation with Multimodal Transformerscited by 135
  19. 2021Few-Shot Segmentation via Cycle-Consistent Transformercited by 133
  20. 2022Pushing the Limits of Simple Pipelines for Few-Shot Learning: External Data and Fine-Tuning Make a Differencecited by 123
  21. 2021Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformercited by 116
  22. 2022Cost Aggregation with 4D Convolutional Swin Transformer for Few-Shot Segmentationcited by 114
  23. 2022Self-Support Few-Shot Semantic Segmentationcited by 96
  24. 2023Referring Multi-Object Trackingcited by 91
  25. 2018Memory Matching Networks for One-Shot Image Recognitioncited by 65
  26. 2022Dense Cross-Query-and-Support Attention Weighted Mask Aggregation for Few-Shot Segmentationcited by 65
  27. 2023OnlineRefer: A Simple Online Baseline for Referring Video Object Segmentationcited by 62
  28. 2022Learning Non-target Knowledge for Few-shot Semantic Segmentationcited by 57
  29. 2021Learning Meta-class Memory for Few-Shot Semantic Segmentationcited by 56
  30. 2022Instance-Specific Feature Propagation for Referring Segmentationcited by 45
  31. 2022Dynamic Prototype Convolution Network for Few-Shot Semantic Segmentationcited by 43
  32. 2022ProposalCLIP: Unsupervised Open-Category Object Proposal Generation via Exploiting CLIP Cuescited by 42
  33. 2023CLIP-S4: Language-Guided Self-Supervised Semantic Segmentationcited by 34
  34. 2023FECANet: Boosting Few-Shot Semantic Segmentation with Feature-Enhanced Context-Aware Networkcited by 22
  35. 2023Delving into Shape-aware Zero-shot Semantic Segmentationcited by 16
  36. 2021Squeezing Backbone Feature Distributions to the Max for Efficient Few-Shot Learningcited by 14
  37. 2022Multi-level Second-order Few-shot Learningcited by 12
  38. 2023Iterative Few-shot Semantic Segmentation from Image Label Textcited by 10

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 7; 2026: 3.

19 papers.

Sort:

By area:

  1. 2024Image Segmentation in Foundation Model Era: A Surveycited by 35
  2. 2024Adapt Before Comparison: A New Perspective on Cross-Domain Few-Shot Segmentationcited by 22
  3. 2024Rethinking Prior Information Generation with CLIP for Few-Shot Segmentationcited by 20
  4. 2024A Novel Benchmark for Few-Shot Semantic Segmentation in the Era of Foundation Modelscited by 13
  5. 2025Not All Data Are Unlearned Equallycited by 12
  6. 2025Vision-Language Model for Object Detection and Segmentation: A Review and Evaluationcited by 10
  7. 2024Group-On: Boosting One-Shot Segmentation with Supportive Querycited by 3
  8. 2024Beyond Mask: Rethinking Guidance Types in Few-shot Segmentationcited by 3
  9. 2024High-Performance Few-Shot Segmentation with Foundation Models: An Empirical Studycited by 3
  10. 2024SAMIC: Segment Anything with In-Context Spatial Prompt Engineeringcited by 3
  11. 2025UINO-FSS: Unifying Representation Learning and Few-shot Segmentation via Hierarchical Distillation and Mamba-HyperCorrelationcited by 1
  12. 2025ViRefSAM: Visual Reference-Guided Segment Anything Model for Remote Sensing Segmentationcited by 1
7 more
  1. 2025Unbiased Semantic Decoding with Vision Foundation Models for Few-shot Segmentationcited by 1
  2. 2024Optimization of Prompt Learning via Multi-Knowledge Representation for Vision-Language Modelscited by 0
  3. 2025Multi-Text Guided Few-Shot Semantic Segmentationcited by 0
  4. 2025Beyond Visual Cues: Leveraging General Semantics as Support for Few-Shot Segmentationcited by 0
  5. 2026Language Prompt vs. Image Enhancement: Boosting Object Detection With CLIP in Hazy Environmentscited by 0
  6. 2026GeoFlow: Geo-Aware Modeling of Inter-Area Relationships in Origin-Destination Flow Prediction and Generationcited by 0
  7. 2026Exploiting Target Knowledge from MLLMs for Robust Few-Shot Segmentationcited by 0