# Unbiased Semantic Decoding with Vision Foundation Models for Few-shot Segmentation

arXiv: https://arxiv.org/abs/2511.15118 · HTML: https://mlsanity.com/paper/2511.15118/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 78 references that are in the map.
- First posted on arXiv 19 Nov 2025.

## Builds on (most cited first, 30 of 45)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [One-Shot Learning for Semantic Segmentation](https://mlsanity.com/paper/1709.03410/) (2017)
- [Prior Guided Feature Enrichment Network for Few-Shot Segmentation](https://mlsanity.com/paper/2008.01449/) (2020)
- [Personalize Segment Anything Model with One Shot](https://mlsanity.com/paper/2305.03048/) (2023)
- [CANet: Class-Agnostic Segmentation Networks with Iterative Refinement and Attentive Few-Shot Learning](https://mlsanity.com/paper/1903.02351/) (2019)
- [Feature Weighting and Boosting for Few-Shot Segmentation](https://mlsanity.com/paper/1909.13140/) (2019)
- [SG-One: Similarity Guidance Network for One-Shot Semantic Segmentation](https://mlsanity.com/paper/1810.09091/) (2018)
- [Segment Anything Model (SAM) for Digital Pathology: Assess Zero-shot Segmentation on Whole Slide Imaging](https://mlsanity.com/paper/2304.04155/) (2023)
- [Learning What Not to Segment: A New Perspective on Few-Shot Segmentation](https://mlsanity.com/paper/2203.07615/) (2022)
- [CLIP is Also an Efficient Segmenter: A Text-Driven Approach for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2212.09506/) (2022)
- [Cost Aggregation with 4D Convolutional Swin Transformer for Few-Shot Segmentation](https://mlsanity.com/paper/2207.10866/) (2022)
- [Large-scale Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2106.03149/) (2021)
- [Self-Support Few-Shot Semantic Segmentation](https://mlsanity.com/paper/2207.11549/) (2022)
- [T-Rex2: Towards Generic Object Detection via Text-Visual Prompt Synergy](https://mlsanity.com/paper/2403.14610/) (2024)
- [Self-Guided and Cross-Guided Learning for Few-Shot Segmentation](https://mlsanity.com/paper/2103.16129/) (2021)
- [Hierarchical Dense Correlation Distillation for Few-Shot Segmentation](https://mlsanity.com/paper/2303.14652/) (2023)
- [VRP-SAM: SAM with Visual Reference Prompt](https://mlsanity.com/paper/2402.17726/) (2024)
- [Dense Cross-Query-and-Support Attention Weighted Mask Aggregation for Few-Shot Segmentation](https://mlsanity.com/paper/2207.08549/) (2022)
- [SAM.MD: Zero-shot medical image segmentation capabilities of the Segment Anything Model](https://mlsanity.com/paper/2304.05396/) (2023)
- [Learning Non-target Knowledge for Few-shot Semantic Segmentation](https://mlsanity.com/paper/2205.04903/) (2022)
- [ClipSAM: CLIP and SAM Collaboration for Zero-Shot Anomaly Segmentation](https://mlsanity.com/paper/2401.12665/) (2024)
- [LLaFS: When Large Language Models Meet Few-Shot Segmentation](https://mlsanity.com/paper/2311.16926/) (2023)
- [Dynamic Prototype Convolution Network for Few-Shot Semantic Segmentation](https://mlsanity.com/paper/2204.10638/) (2022)
- [Intermediate Prototype Mining Transformer for Few-Shot Semantic Segmentation](https://mlsanity.com/paper/2210.06780/) (2022)
- [PFENet++: Boosting Few-shot Semantic Segmentation with the Noise-filtered Context-aware Prior Mask](https://mlsanity.com/paper/2109.13788/) (2021)
- [BriNet: Towards Bridging the Intra-class and Inter-class Gaps in One-Shot Segmentation](https://mlsanity.com/paper/2008.06226/) (2020)
- [CAT-SAM: Conditional Tuning for Few-Shot Adaptation of Segment Anything Model](https://mlsanity.com/paper/2402.03631/) (2024)

## Built on by (most cited first, 1 of 1)

- [Beyond Visual Cues: Leveraging General Semantics as Support for Few-Shot Segmentation](https://mlsanity.com/paper/2511.16435/) (2025)

## Abstract (arXiv)

Few-shot segmentation has garnered significant attention. Many recent approaches attempt to introduce the Segment Anything Model (SAM) to handle this task. With the strong generalization ability and rich object-specific extraction ability of the SAM model, such a solution shows great potential in few-shot segmentation. However, the decoding process of SAM highly relies on accurate and explicit prompts, making previous approaches mainly focus on extracting prompts from the support set, which is insufficient to activate the generalization ability of SAM, and this design is easy to result in a biased decoding process when adapting to the unknown classes. In this work, we propose an Unbiased Semantic Decoding (USD) strategy integrated with SAM, which extracts target information from both the support and query set simultaneously to perform consistent predictions guided by the semantics of the Contrastive Language-Image Pre-training (CLIP) model. Specifically, to enhance the unbiased semantic discrimination of SAM, we design two feature enhancement strategies that leverage the semantic alignment capability of CLIP to enrich the original SAM features, mainly including a global supplement at the image level to provide a generalize category indicate with support image and a local guidance at the pixel level to provide a useful target location with query image. Besides, to generate target-focused prompt embeddings, a learnable visual-text target prompt generator is proposed by interacting target text embeddings and clip visual features. Without requiring re-training of the vision foundation models, the features with semantic discrimination draw attention to the target region through the guidance of prompt with rich target information.