# Rethinking Prior Information Generation with CLIP for Few-Shot Segmentation

arXiv: https://arxiv.org/abs/2405.08458 · HTML: https://mlsanity.com/paper/2405.08458/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 63 references that are in the map.
- First posted on arXiv 14 May 2024.

## Builds on (most cited first, 30 of 50)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [Image Segmentation Using Deep Learning: A Survey](https://mlsanity.com/paper/2001.05566/) (2020)
- [Image Segmentation Using Text and Image Prompts](https://mlsanity.com/paper/2112.10003/) (2021)
- [PANet: Few-Shot Image Semantic Segmentation with Prototype Alignment](https://mlsanity.com/paper/1908.06391/) (2019)
- [Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classification](https://mlsanity.com/paper/2207.09519/) (2022)
- [One-Shot Learning for Semantic Segmentation](https://mlsanity.com/paper/1709.03410/) (2017)
- [Prior Guided Feature Enrichment Network for Few-Shot Segmentation](https://mlsanity.com/paper/2008.01449/) (2020)
- [Adaptive Prototype Learning and Allocation for Few-Shot Segmentation](https://mlsanity.com/paper/2104.01893/) (2021)
- [Feature Weighting and Boosting for Few-Shot Segmentation](https://mlsanity.com/paper/1909.13140/) (2019)
- [CLIP2Point: Transfer CLIP to Point Cloud Classification with Image-Depth Pre-training](https://mlsanity.com/paper/2210.01055/) (2022)
- [Learning What Not to Segment: A New Perspective on Few-Shot Segmentation](https://mlsanity.com/paper/2203.07615/) (2022)
- [CLIP is Also an Efficient Segmenter: A Text-Driven Approach for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2212.09506/) (2022)
- [Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformer](https://mlsanity.com/paper/2108.03032/) (2021)
- [Cost Aggregation with 4D Convolutional Swin Transformer for Few-Shot Segmentation](https://mlsanity.com/paper/2207.10866/) (2022)
- [CALIP: Zero-Shot Enhancement of CLIP with Parameter-free Attention](https://mlsanity.com/paper/2209.14169/) (2022)
- [Self-Support Few-Shot Semantic Segmentation](https://mlsanity.com/paper/2207.11549/) (2022)
- [Self-Guided and Cross-Guided Learning for Few-Shot Segmentation](https://mlsanity.com/paper/2103.16129/) (2021)
- [Hierarchical Dense Correlation Distillation for Few-Shot Segmentation](https://mlsanity.com/paper/2303.14652/) (2023)
- [Not All Features Matter: Enhancing Few-shot CLIP with Adaptive Prior Refinement](https://mlsanity.com/paper/2304.01195/) (2023)
- [Dense Cross-Query-and-Support Attention Weighted Mask Aggregation for Few-Shot Segmentation](https://mlsanity.com/paper/2207.08549/) (2022)
- [Learning Non-target Knowledge for Few-shot Semantic Segmentation](https://mlsanity.com/paper/2205.04903/) (2022)
- [Learning Mask-aware CLIP Representations for Zero-Shot Segmentation](https://mlsanity.com/paper/2310.00240/) (2023)
- [Generalized Parametric Contrastive Learning](https://mlsanity.com/paper/2209.12400/) (2022)
- [Dynamic Prototype Convolution Network for Few-Shot Semantic Segmentation](https://mlsanity.com/paper/2204.10638/) (2022)
- [PFENet++: Boosting Few-shot Semantic Segmentation with the Noise-filtered Context-aware Prior Mask](https://mlsanity.com/paper/2109.13788/) (2021)
- [Few-Shot Segmentation Propagation with Guided Networks](https://mlsanity.com/paper/1806.07373/) (2018)

## Built on by (most cited first, 20 of 20)

- [Image Segmentation in Foundation Model Era: A Survey](https://mlsanity.com/paper/2408.12957/) (2024)
- [High-Performance Few-Shot Segmentation with Foundation Models: An Empirical Study](https://mlsanity.com/paper/2409.06305/) (2024)
- [NeuroSeg Meets DINOv3: Transferring 2D Self-Supervised Visual Priors to 3D Neuron Segmentation via DINOv3 Initialization](https://mlsanity.com/paper/2603.23104/) (2026)
- [AFANet: Adaptive Frequency-Aware Network for Weakly-Supervised Few-Shot Semantic Segmentation](https://mlsanity.com/paper/2412.17601/) (2024)
- [MARS: a Multimodal Alignment and Ranking System for Few-Shot Segmentation](https://mlsanity.com/paper/2504.07942/) (2025)
- [UINO-FSS: Unifying Representation Learning and Few-shot Segmentation via Hierarchical Distillation and Mamba-HyperCorrelation](https://mlsanity.com/paper/2504.15669/) (2025)
- [Probabilistic Prototype Calibration of Vision-Language Models for Generalized Few-shot Semantic Segmentation](https://mlsanity.com/paper/2506.22979/) (2025)
- [ViRefSAM: Visual Reference-Guided Segment Anything Model for Remote Sensing Segmentation](https://mlsanity.com/paper/2507.02294/) (2025)
- [Balancing Conservatism and Aggressiveness: Prototype-Affinity Hybrid Network for Few-Shot Segmentation](https://mlsanity.com/paper/2507.19140/) (2025)
- [Training-free Cross-domain Few-shot Segmentation via Robust Semantic Representation and Matching](https://mlsanity.com/paper/2606.24297/) (2026)
- [FCC: Fully Connected Correlation for One-Shot Segmentation](https://mlsanity.com/paper/2411.11917/) (2024)
- [Learning Attribute-aware Representations for Few-shot Scene Text Segmentation](https://mlsanity.com/paper/2504.11164/) (2025)
- [Efficient Masked Attention Transformer for Few-Shot Classification and Segmentation](https://mlsanity.com/paper/2507.23642/) (2025)
- [Multi-Text Guided Few-Shot Semantic Segmentation](https://mlsanity.com/paper/2511.15515/) (2025)
- [Beyond Visual Cues: Leveraging General Semantics as Support for Few-Shot Segmentation](https://mlsanity.com/paper/2511.16435/) (2025)
- [Language Prompt vs. Image Enhancement: Boosting Object Detection With CLIP in Hazy Environments](https://mlsanity.com/paper/2604.10637/) (2026)
- [Hierarchical Spatial and Channel Aggregation for Cross-domain Few-shot Segmentation](https://mlsanity.com/paper/2606.24296/) (2026)
- [GeoFlow: Geo-Aware Modeling of Inter-Area Relationships in Origin-Destination Flow Prediction and Generation](https://mlsanity.com/paper/2607.05257/) (2026)
- [P3-SAM: SAM with Perceptual Parallel Prompt for Few-Shot Strip Steel Surface Defect Segmentation](https://mlsanity.com/paper/2609.21424/) (2026)
- [Exploiting Target Knowledge from MLLMs for Robust Few-Shot Segmentation](https://mlsanity.com/paper/2609.28949/) (2026)

## Abstract (arXiv)

Few-shot segmentation remains challenging due to the limitations of its labeling information for unseen classes. Most previous approaches rely on extracting high-level feature maps from the frozen visual encoder to compute the pixel-wise similarity as a key prior guidance for the decoder. However, such a prior representation suffers from coarse granularity and poor generalization to new classes since these high-level feature maps have obvious category bias. In this work, we propose to replace the visual prior representation with the visual-text alignment capacity to capture more reliable guidance and enhance the model generalization. Specifically, we design two kinds of training-free prior information generation strategy that attempts to utilize the semantic alignment capability of the Contrastive Language-Image Pre-training model (CLIP) to locate the target class. Besides, to acquire more accurate prior guidance, we build a high-order relationship of attention maps and utilize it to refine the initial prior information. Experiments on both the PASCAL-5{i} and COCO-20{i} datasets show that our method obtains a clearly substantial improvement and reaches the new state-of-the-art performance.