# SAM-PARSER: Fine-tuning SAM Efficiently by Parameter Space Reconstruction

arXiv: https://arxiv.org/abs/2308.14604 · HTML: https://mlsanity.com/paper/2308.14604/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 41 references that are in the map.
- First posted on arXiv 28 Aug 2023.

## Builds on (most cited first, 26 of 26)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning](https://mlsanity.com/paper/2012.13255/) (2020)
- [Segment Everything Everywhere All at Once](https://mlsanity.com/paper/2304.06718/) (2023)
- [Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.12620/) (2023)
- [SVDiff: Compact Parameter Space for Diffusion Fine-Tuning](https://mlsanity.com/paper/2303.11305/) (2023)
- [Customized Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.13785/) (2023)
- [AbdomenCT-1K: Is Abdominal Organ Segmentation A Solved Problem?](https://mlsanity.com/paper/2010.14808/) (2020)
- [SegGPT: Segmenting Everything In Context](https://mlsanity.com/paper/2304.03284/) (2023)
- [FacT: Factor-Tuning for Lightweight Adaptation on Vision Transformer](https://mlsanity.com/paper/2212.03145/) (2022)
- [A Survey on Object Detection in Optical Remote Sensing Images](https://mlsanity.com/paper/1603.06201/) (2016)
- [SAM Struggles in Concealed Scenes -- Empirical Study on Segment Anything](https://mlsanity.com/paper/2304.06022/) (2023)
- [Polyp-SAM: Transfer SAM for Polyp Segmentation](https://mlsanity.com/paper/2305.00293/) (2023)
- [Parameter-Efficient Fine-Tuning for Medical Image Analysis: The Missed Opportunity](https://mlsanity.com/paper/2305.08252/) (2023)
- [Singular Value Fine-tuning: Few-shot Segmentation requires Few-parameters Fine-tuning](https://mlsanity.com/paper/2206.06122/) (2022)
- [Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator](https://mlsanity.com/paper/2305.15099/) (2023)
- [Detect Any Shadow: Segment Anything for Video Shadow Detection](https://mlsanity.com/paper/2305.16698/) (2023)
- [High-dimensional vector autoregressive time series modeling via tensor decomposition](https://mlsanity.com/paper/1909.06624/) (2019)
- [When SAM Meets Sonar Images](https://mlsanity.com/paper/2306.14109/) (2023)
- [BadSAM: Exploring Security Vulnerabilities of SAM via Backdoor Attacks](https://mlsanity.com/paper/2305.03289/) (2023)

## Built on by (most cited first, 23 of 23)

- [SVFT: Parameter-Efficient Fine-Tuning with Singular Vectors](https://mlsanity.com/paper/2405.19597/) (2024)
- [SAGD: Boundary-Enhanced Segment Anything in 3D Gaussian via Gaussian Decomposition](https://mlsanity.com/paper/2401.17857/) (2024)
- [Parameter Efficient Fine-tuning via Cross Block Orchestration for Segment Anything Model](https://mlsanity.com/paper/2311.17112/) (2023)
- [See Further for Parameter Efficient Fine-tuning by Standing on the Shoulders of Decomposition](https://mlsanity.com/paper/2407.05417/) (2024)
- [Improving SAM for Camouflaged Object Detection via Dual Stream Adapters](https://mlsanity.com/paper/2503.06042/) (2025)
- [ASAM: Boosting Segment Anything Model with Adversarial Tuning](https://mlsanity.com/paper/2405.00256/) (2024)
- [SAM-OCTA: Prompting Segment-Anything for OCTA Image Segmentation](https://mlsanity.com/paper/2310.07183/) (2023)
- [FINE: Factorizing Knowledge for Initialization of Variable-sized Diffusion Models](https://mlsanity.com/paper/2409.19289/) (2024)
- [Quantifying the Limits of Segmentation Foundation Models: Modeling Challenges in Segmenting Tree-Like and Low-Contrast Objects](https://mlsanity.com/paper/2412.04243/) (2024)
- [MoORE: SVD-based Model MoE-ization for Conflict- and Oblivion-Resistant Multi-Task Adaptation](https://mlsanity.com/paper/2506.14436/) (2025)
- [Multi-scale Contrastive Adaptor Learning for Segmenting Anything in Underperformed Scenes](https://mlsanity.com/paper/2408.05936/) (2024)
- [CLIP-SVD: Efficient and Interpretable Vision-Language Adaptation via Singular Values](https://mlsanity.com/paper/2509.03740/) (2025)
- [TransLandSeg: A Transfer Learning Approach for Landslide Semantic Segmentation Based on Vision Foundation Model](https://mlsanity.com/paper/2403.10127/) (2024)
- [Semantic Segmentation Refiner for Ultrasound Applications with Zero-Shot Foundation Models](https://mlsanity.com/paper/2404.16325/) (2024)
- [Supervise Less, See More: Training-free Nuclear Instance Segmentation with Prototype-Guided Prompting](https://mlsanity.com/paper/2511.19953/) (2025)
- [A Survey of Low-shot Vision-Language Model Adaptation via Representer Theorem](https://mlsanity.com/paper/2410.11686/) (2024)
- [BALD-SAM: Disagreement-based Active Prompting in Interactive Segmentation](https://mlsanity.com/paper/2603.10828/) (2026)
- [Parameter-Free Fine-tuning via Redundancy Elimination for Vision Foundation Models](https://mlsanity.com/paper/2504.08915/) (2025)
- [Multi-Sequence Parotid Gland Lesion Segmentation via Expert Text-Guided Segment Anything Model](https://mlsanity.com/paper/2508.09645/) (2025)
- [Quickly Tuning Foundation Models for Image Segmentation](https://mlsanity.com/paper/2508.17283/) (2025)
- [Efficient Multi-Source Knowledge Transfer by Model Merging](https://mlsanity.com/paper/2508.19353/) (2025)
- [Attack for Defense: Adversarial Agents for Point Prompt Optimization Empowering Segment Anything Model](https://mlsanity.com/paper/2509.18891/) (2025)
- [Amped: Adaptive Multi-stage Non-edge Pruning for Edge Detection](https://mlsanity.com/paper/2603.27661/) (2026)

## Abstract (arXiv)

Segment Anything Model (SAM) has received remarkable attention as it offers a powerful and versatile solution for object segmentation in images. However, fine-tuning SAM for downstream segmentation tasks under different scenarios remains a challenge, as the varied characteristics of different scenarios naturally requires diverse model parameter spaces. Most existing fine-tuning methods attempt to bridge the gaps among different scenarios by introducing a set of new parameters to modify SAM's original parameter space. Unlike these works, in this paper, we propose fine-tuning SAM efficiently by parameter space reconstruction (SAM-PARSER), which introduce nearly zero trainable parameters during fine-tuning. In SAM-PARSER, we assume that SAM's original parameter space is relatively complete, so that its bases are able to reconstruct the parameter space of a new scenario. We obtain the bases by matrix decomposition, and fine-tuning the coefficients to reconstruct the parameter space tailored to the new scenario by an optimal linear combination of the bases. Experimental results show that SAM-PARSER exhibits superior segmentation performance across various scenarios, while reducing the number of trainable parameters by ≈ 290 times compared with current parameter-efficient fine-tuning methods.