# Unlocking the Power of SAM 2 for Few-Shot Segmentation

arXiv: https://arxiv.org/abs/2505.14100 · HTML: https://mlsanity.com/paper/2505.14100/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 43 references that are in the map.
- First posted on arXiv 20 May 2025.

## Builds on (most cited first, 30 of 36)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [PANet: Few-Shot Image Semantic Segmentation with Prototype Alignment](https://mlsanity.com/paper/1908.06391/) (2019)
- [One-Shot Learning for Semantic Segmentation](https://mlsanity.com/paper/1709.03410/) (2017)
- [Prior Guided Feature Enrichment Network for Few-Shot Segmentation](https://mlsanity.com/paper/2008.01449/) (2020)
- [Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Parts](https://mlsanity.com/paper/1406.2031/) (2014)
- [Feature Weighting and Boosting for Few-Shot Segmentation](https://mlsanity.com/paper/1909.13140/) (2019)
- [Simultaneous Detection and Segmentation](https://mlsanity.com/paper/1407.1808/) (2014)
- [Matcher: Segment Anything with One Shot Using All-Purpose Feature Matching](https://mlsanity.com/paper/2305.13310/) (2023)
- [Few-Shot Segmentation via Cycle-Consistent Transformer](https://mlsanity.com/paper/2106.02320/) (2021)
- [Cost Aggregation with 4D Convolutional Swin Transformer for Few-Shot Segmentation](https://mlsanity.com/paper/2207.10866/) (2022)
- [Self-Support Few-Shot Semantic Segmentation](https://mlsanity.com/paper/2207.11549/) (2022)
- [Hierarchical Dense Correlation Distillation for Few-Shot Segmentation](https://mlsanity.com/paper/2303.14652/) (2023)
- [VRP-SAM: SAM with Visual Reference Prompt](https://mlsanity.com/paper/2402.17726/) (2024)
- [Dense Cross-Query-and-Support Attention Weighted Mask Aggregation for Few-Shot Segmentation](https://mlsanity.com/paper/2207.08549/) (2022)
- [LLaFS: When Large Language Models Meet Few-Shot Segmentation](https://mlsanity.com/paper/2311.16926/) (2023)
- [Bridge the Points: Graph-based Few-shot Segment Anything Semantically](https://mlsanity.com/paper/2410.06964/) (2024)
- [Self-Calibrated Cross Attention Network for Few-Shot Segmentation](https://mlsanity.com/paper/2308.09294/) (2023)
- [Doubly Deformable Aggregation of Covariance Matrices for Few-shot Segmentation](https://mlsanity.com/paper/2208.00306/) (2022)
- [FECANet: Boosting Few-Shot Semantic Segmentation with Feature-Enhanced Context-Aware Network](https://mlsanity.com/paper/2301.08160/) (2023)
- [MSANet: Multi-Similarity and Attention Guidance for Boosting Few-Shot Segmentation](https://mlsanity.com/paper/2206.09667/) (2022)
- [Hybrid Mamba for Few-Shot Segmentation](https://mlsanity.com/paper/2409.19613/) (2024)
- [Eliminating Feature Ambiguity for Few-Shot Segmentation](https://mlsanity.com/paper/2407.09842/) (2024)

## Built on by (most cited first, 11 of 11)

- [Atlas is Your Perfect Context: One-Shot Customization for Generalizable Foundational Medical Image Segmentation](https://mlsanity.com/paper/2512.18176/) (2025)
- [Few-Shot Semantic Segmentation Meets SAM3](https://mlsanity.com/paper/2604.05433/) (2026)
- [Omni-Referring Image Segmentation](https://mlsanity.com/paper/2512.06862/) (2025)
- [Boosting SAM for Cross-Domain Few-Shot Segmentation via Conditional Point Sparsification](https://mlsanity.com/paper/2602.05218/) (2026)
- [Make It Up: Fake Images, Real Gains in Generalized Few-shot Semantic Segmentation](https://mlsanity.com/paper/2603.27206/) (2026)
- [Training-free Cross-domain Few-shot Segmentation via Robust Semantic Representation and Matching](https://mlsanity.com/paper/2606.24297/) (2026)
- [Inter- and Intra-image Refinement for Few Shot Segmentation](https://mlsanity.com/paper/2507.05838/) (2025)
- [Adaptive Attention Distillation for Robust Few-Shot Segmentation under Environmental Perturbations](https://mlsanity.com/paper/2601.03596/) (2026)
- [Training-Free Generalized Few-Shot Segmentation through Open-Vocabulary Semantic Arbitration](https://mlsanity.com/paper/2606.09474/) (2026)
- [Hierarchical Spatial and Channel Aggregation for Cross-domain Few-shot Segmentation](https://mlsanity.com/paper/2606.24296/) (2026)
- [Exploiting Target Knowledge from MLLMs for Robust Few-Shot Segmentation](https://mlsanity.com/paper/2609.28949/) (2026)

## Abstract (arXiv)

Few-Shot Segmentation (FSS) aims to learn class-agnostic segmentation on few classes to segment arbitrary classes, but at the risk of overfitting. To address this, some methods use the well-learned knowledge of foundation models (e.g., SAM) to simplify the learning process. Recently, SAM 2 has extended SAM by supporting video segmentation, whose class-agnostic matching ability is useful to FSS. A simple idea is to encode support foreground (FG) features as memory, with which query FG features are matched and fused. Unfortunately, the FG objects in different frames of SAM 2's video data are always the same identity, while those in FSS are different identities, i.e., the matching step is incompatible. Therefore, we design Pseudo Prompt Generator to encode pseudo query memory, matching with query features in a compatible way. However, the memories can never be as accurate as the real ones, i.e., they are likely to contain incomplete query FG, and some unexpected query background (BG) features, leading to wrong segmentation. Hence, we further design Iterative Memory Refinement to fuse more query FG features into the memory, and devise a Support-Calibrated Memory Attention to suppress the unexpected query BG features in memory. Extensive experiments have been conducted on PASCAL-5i and COCO-20i to validate the effectiveness of our design, e.g., the 1-shot mIoU can be 4.2% better than the best baseline.