# FS-SAM2: Adapting Segment Anything Model 2 for Few-Shot Semantic Segmentation via Low-Rank Adaptation

arXiv: https://arxiv.org/abs/2509.12105 · HTML: https://mlsanity.com/paper/2509.12105/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 35 references that are in the map.
- First posted on arXiv 15 Sep 2025.

## Builds on (most cited first, 29 of 29)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [PANet: Few-Shot Image Semantic Segmentation with Prototype Alignment](https://mlsanity.com/paper/1908.06391/) (2019)
- [One-Shot Learning for Semantic Segmentation](https://mlsanity.com/paper/1709.03410/) (2017)
- [Hiera: A Hierarchical Vision Transformer without the Bells-and-Whistles](https://mlsanity.com/paper/2306.00989/) (2023)
- [Prior Guided Feature Enrichment Network for Few-Shot Segmentation](https://mlsanity.com/paper/2008.01449/) (2020)
- [Hypercorrelation Squeeze for Few-Shot Segmentation](https://mlsanity.com/paper/2104.01538/) (2021)
- [Feature Weighting and Boosting for Few-Shot Segmentation](https://mlsanity.com/paper/1909.13140/) (2019)
- [FSS-1000: A 1000-Class Dataset for Few-Shot Segmentation](https://mlsanity.com/paper/1907.12347/) (2019)
- [Medical SAM 2: Segment medical images as video via Segment Anything Model 2](https://mlsanity.com/paper/2408.00874/) (2024)
- [Matcher: Segment Anything with One Shot Using All-Purpose Feature Matching](https://mlsanity.com/paper/2305.13310/) (2023)
- [Few-Shot Segmentation via Cycle-Consistent Transformer](https://mlsanity.com/paper/2106.02320/) (2021)
- [Cost Aggregation with 4D Convolutional Swin Transformer for Few-Shot Segmentation](https://mlsanity.com/paper/2207.10866/) (2022)
- [Low-Rank Few-Shot Adaptation of Vision-Language Models](https://mlsanity.com/paper/2405.18541/) (2024)
- [Hierarchical Dense Correlation Distillation for Few-Shot Segmentation](https://mlsanity.com/paper/2303.14652/) (2023)
- [VRP-SAM: SAM with Visual Reference Prompt](https://mlsanity.com/paper/2402.17726/) (2024)
- [Dense Cross-Query-and-Support Attention Weighted Mask Aggregation for Few-Shot Segmentation](https://mlsanity.com/paper/2207.08549/) (2022)
- [Feature-Proxy Transformer for Few-Shot Segmentation](https://mlsanity.com/paper/2210.06908/) (2022)
- [Singular Value Fine-tuning: Few-shot Segmentation requires Few-parameters Fine-tuning](https://mlsanity.com/paper/2206.06122/) (2022)
- [Bridge the Points: Graph-based Few-shot Segment Anything Semantically](https://mlsanity.com/paper/2410.06964/) (2024)
- [Self-Calibrated Cross Attention Network for Few-Shot Segmentation](https://mlsanity.com/paper/2308.09294/) (2023)
- [Semantic Image Segmentation: Two Decades of Research](https://mlsanity.com/paper/2302.06378/) (2023)
- [A Novel Benchmark for Few-Shot Semantic Segmentation in the Era of Foundation Models](https://mlsanity.com/paper/2401.11311/) (2024)
- [RevSAM2: Prompt SAM2 for Medical Image Segmentation via Reverse-Propagation without Fine-tuning](https://mlsanity.com/paper/2409.04298/) (2024)
- [MSDNet: Multi-Scale Decoder for Few-Shot Semantic Segmentation via Transformer-Guided Prototyping](https://mlsanity.com/paper/2409.11316/) (2024)

## Built on by (most cited first, 1 of 1)

- [FoundYou: A Unified Model for Personalized Segmentation and Retrieval](https://mlsanity.com/paper/2608.29917/) (2026)

## Abstract (arXiv)

Few-shot semantic segmentation has recently attracted great attention. The goal is to develop a model capable of segmenting unseen classes using only a few annotated samples. Most existing approaches adapt a pre-trained model by training from scratch an additional module. Achieving optimal performance with these approaches requires extensive training on large-scale datasets. The Segment Anything Model 2 (SAM2) is a foundational model for zero-shot image and video segmentation with a modular design. In this paper, we propose a Few-Shot segmentation method based on SAM2 (FS-SAM2), where SAM2's video capabilities are directly repurposed for the few-shot task. Moreover, we apply a Low-Rank Adaptation (LoRA) to the original modules in order to handle the diverse images typically found in standard datasets, unlike the temporally connected frames used in SAM2's pre-training. With this approach, only a small number of parameters is meta-trained, which effectively adapts SAM2 while benefiting from its impressive segmentation performance. Our method supports any K-shot configuration. We evaluate FS-SAM2 on the PASCAL-5i, COCO-20i and FSS-1000 datasets, achieving remarkable results and demonstrating excellent computational efficiency during inference. Code is available at https://github.com/fornib/FS-SAM2