Unbiased Semantic Decoding with Vision Foundation Models for Few-shot Segmentation
Not on the map. Why?
Abstract (arXiv)
Few-shot segmentation has garnered significant attention. Many recent approaches attempt to introduce the Segment Anything Model (SAM) to handle this task. With the strong generalization ability and rich object-specific extraction ability of the SAM model, such a solution shows great potential in few-shot segmentation. However, the decoding process of SAM highly relies on accurate and explicit prompts, making previous approaches mainly focus on extracting prompts from the support set, which is insufficient to activate the generalization ability of SAM, and this design is easy to result in a biased decoding process when adapting to the unknown classes. In this work, we propose an Unbiased Semantic Decoding (USD) strategy integrated with SAM, which extracts target information from both the support and query set simultaneously to perform consistent predictions guided by the semantics of the Contrastive Language-Image Pre-training (CLIP) model. Specifically, to enhance the unbiased semantic discrimination of SAM, we design two feature enhancement strategies that leverage the semantic alignment capability of CLIP to enrich the original SAM features, mainly including a global supplement at the image level to provide a generalize category indicate with support image and a local guidance at the pixel level to provide a useful target location with query image. Besides, to generate target-focused prompt embeddings, a learnable visual-text target prompt generator is proposed by interacting target text embeddings and clip visual features. Without requiring re-training of the vision foundation models, the features with semantic discrimination draw attention to the target region through the guidance of prompt with rich target information.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 45 of its 78 references that are in the map. First posted on arXiv 19 Nov 2025.
Builds on
45 of 78 references.
References · 45
- 2017Attention Is All You Needcited by 60,361
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2023Segment Anythingcited by 8,234
- 2017One-Shot Learning for Semantic Segmentationcited by 337
- 2020Prior Guided Feature Enrichment Network for Few-Shot Segmentationcited by 271
- 2023Personalize Segment Anything Model with One Shotcited by 250
- 2019CANet: Class-Agnostic Segmentation Networks with Iterative Refinement and Attentive Few-Shot Learningcited by 217
- 2019Feature Weighting and Boosting for Few-Shot Segmentationcited by 189
- 2018SG-One: Similarity Guidance Network for One-Shot Semantic Segmentationcited by 169
- 2023Segment Anything Model (SAM) for Digital Pathology: Assess Zero-shot Segmentation on Whole Slide Imagingcited by 161
- 2022Learning What Not to Segment: A New Perspective on Few-Shot Segmentationcited by 120
33 more
- 2022CLIP is Also an Efficient Segmenter: A Text-Driven Approach for Weakly Supervised Semantic Segmentationcited by 118
- 2022Cost Aggregation with 4D Convolutional Swin Transformer for Few-Shot Segmentationcited by 114
- 2021Large-scale Unsupervised Semantic Segmentationcited by 101
- 2022Self-Support Few-Shot Semantic Segmentationcited by 96
- 2024T-Rex2: Towards Generic Object Detection via Text-Visual Prompt Synergycited by 94
- 2021Self-Guided and Cross-Guided Learning for Few-Shot Segmentationcited by 92
- 2023Hierarchical Dense Correlation Distillation for Few-Shot Segmentationcited by 84
- 2024VRP-SAM: SAM with Visual Reference Promptcited by 75
- 2022Dense Cross-Query-and-Support Attention Weighted Mask Aggregation for Few-Shot Segmentationcited by 65
- 2023SAM.MD: Zero-shot medical image segmentation capabilities of the Segment Anything Modelcited by 61
- 2022Learning Non-target Knowledge for Few-shot Semantic Segmentationcited by 57
- 2024ClipSAM: CLIP and SAM Collaboration for Zero-Shot Anomaly Segmentationcited by 49
- 2023LLaFS: When Large Language Models Meet Few-Shot Segmentationcited by 48
- 2022Dynamic Prototype Convolution Network for Few-Shot Semantic Segmentationcited by 43
- 2022Intermediate Prototype Mining Transformer for Few-Shot Semantic Segmentationcited by 37
- 2021PFENet++: Boosting Few-shot Semantic Segmentation with the Noise-filtered Context-aware Prior Maskcited by 31
- 2020BriNet: Towards Bridging the Intra-class and Inter-class Gaps in One-Shot Segmentationcited by 28
- 2024CAT-SAM: Conditional Tuning for Few-Shot Adaptation of Segment Anything Modelcited by 27
- 2024APSeg: Auto-Prompt Network for Cross-Domain Few-Shot Semantic Segmentationcited by 26
- 2023MIANet: Aggregating Unbiased Instance and General Information for Few-Shot Semantic Segmentationcited by 25
- 2023Multi-Modal Prototypes for Open-World Semantic Segmentationcited by 22
- 2024Hybrid Mamba for Few-Shot Segmentationcited by 21
- 2023Visual and Textual Prior Guided Mask Assemble for Few-Shot Segmentation and Beyondcited by 19
- 2024Eliminating Feature Ambiguity for Few-Shot Segmentationcited by 19
- 2021Adaptive Mutual Supervision for Weakly-Supervised Temporal Action Localizationcited by 18
- 2022MSI: Maximize Support-Set Information for Few-Shot Segmentationcited by 18
- 2020Self-Supervised Tuning for Few-Shot Segmentationcited by 17
- 2024Unleashing the Potential of the Diffusion Model in Few-shot Semantic Segmentationcited by 16
- 2023Relevant Intrinsic Feature Enhancement Network for Few-Shot Semantic Segmentationcited by 11
- 2024Prompt-and-Transfer: Dynamic Class-aware Enhancement for Few-shot Segmentationcited by 10
- 2025Foreground-Covering Prototype Generation and Matching for SAM-Aided Few-Shot Segmentationcited by 9
- 2024RevSAM2: Prompt SAM2 for Medical Image Segmentation via Reverse-Propagation without Fine-tuningcited by 7
- 2024AgMTR: Agent Mining Transformer for Few-shot Segmentation in Remote Sensingcited by 6
Built on by
Citing papers per year: 2025: 1.
1 paper.