# TS-SAM: Fine-Tuning Segment-Anything Model for Downstream Tasks

arXiv: https://arxiv.org/abs/2408.01835 · HTML: https://mlsanity.com/paper/2408.01835/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 43 references that are in the map.
- First posted on arXiv 3 Aug 2024.

## Builds on (most cited first, 26 of 26)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [LAVT: Language-Aware Vision Transformer for Referring Image Segmentation](https://mlsanity.com/paper/2112.02244/) (2021)
- [Side Adapter Network for Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2302.12242/) (2023)
- [LST: Ladder Side-Tuning for Parameter and Memory Efficient Transfer Learning](https://mlsanity.com/paper/2206.06522/) (2022)
- [Anabranch Network for Camouflaged Object Segmentation](https://mlsanity.com/paper/2105.09451/) (2021)
- [Concealed Object Detection](https://mlsanity.com/paper/2102.10274/) (2021)
- [Simultaneously Localize, Segment and Rank the Camouflaged Objects](https://mlsanity.com/paper/2103.04011/) (2021)
- [Stacked Conditional Generative Adversarial Networks for Jointly Learning Shadow Detection and Shadow Removal](https://mlsanity.com/paper/1712.02478/) (2017)
- [Zoom In and Out: A Mixed-scale Triplet Network for Camouflaged Object Detection](https://mlsanity.com/paper/2203.02688/) (2022)
- [Salient Object Detection via Integrity Learning](https://mlsanity.com/paper/2101.07663/) (2021)
- [Global Context-Aware Progressive Aggregation Network for Salient Object Detection](https://mlsanity.com/paper/2003.00651/) (2020)
- [Can SAM Segment Anything? When SAM Meets Camouflaged Object Detection](https://mlsanity.com/paper/2304.04709/) (2023)
- [Feature Shrinkage Pyramid for Camouflaged Object Detection with Transformers](https://mlsanity.com/paper/2303.14816/) (2023)
- [High-resolution Iterative Feedback Network for Camouflaged Object Detection](https://mlsanity.com/paper/2203.11624/) (2022)
- [Deep Gradient Learning for Efficient Camouflaged Object Detection](https://mlsanity.com/paper/2205.12853/) (2022)
- [Deep RGB-D Saliency Detection with Depth-Sensitive Attention and Automatic Multi-Modal Fusion](https://mlsanity.com/paper/2103.11832/) (2021)
- [Deep Contrast Learning for Salient Object Detection](https://mlsanity.com/paper/1603.01976/) (2016)
- [Feature Aggregation and Propagation Network for Camouflaged Object Detection](https://mlsanity.com/paper/2212.00990/) (2022)
- [Towards Adaptive Prefix Tuning for Parameter-Efficient Language Model Fine-tuning](https://mlsanity.com/paper/2305.15212/) (2023)
- [Structured Modeling of Joint Deep Feature and Prediction Refinement for Salient Object Detection](https://mlsanity.com/paper/1909.04366/) (2019)
- [SDDNet: Style-guided Dual-layer Disentanglement Network for Shadow Detection](https://mlsanity.com/paper/2308.08935/) (2023)
- [Fine-Context Shadow Detection using Shadow Removal](https://mlsanity.com/paper/2109.09609/) (2021)
- [Hierarchical Side-Tuning for Vision Transformers](https://mlsanity.com/paper/2310.05393/) (2023)
- [SAD: Saliency-based Defenses Against Adversarial Examples](https://mlsanity.com/paper/2003.04820/) (2020)

## Built on by (most cited first, 5 of 5)

- [LSP-ST: Ladder Shape-Biased Side-Tuning for Robust Infrared Small Target Detection](https://mlsanity.com/paper/2504.14481/) (2025)
- [Industrial Synthetic Segment Pre-training](https://mlsanity.com/paper/2505.13099/) (2025)
- [Towards Size-invariant Salient Object Detection: A Generic Evaluation and Optimization Approach](https://mlsanity.com/paper/2509.15573/) (2025)
- [From Local Matches to Global Masks: Template-Guided Instance Detection and Segmentation in Open-World Scenes](https://mlsanity.com/paper/2603.03577/) (2026)
- [CLIP-Guided SAM: Parameter-Efficient Semantic Conditioning for Promptable Segmentation](https://mlsanity.com/paper/2605.24807/) (2026)

## Abstract (arXiv)

Adapter based fine-tuning has been studied for improving the performance of SAM on downstream tasks. However, there is still a significant performance gap between fine-tuned SAMs and domain-specific models. To reduce the gap, we propose Two-Stream SAM (TS-SAM). On the one hand, inspired by the side network in Parameter-Efficient Fine-Tuning (PEFT), we designed a lightweight Convolutional Side Adapter (CSA), which integrates the powerful features from SAM into side network training for comprehensive feature fusion. On the other hand, in line with the characteristics of segmentation tasks, we designed Multi-scale Refinement Module (MRM) and Feature Fusion Decoder (FFD) to keep both the detailed and semantic features. Extensive experiments on ten public datasets from three tasks demonstrate that TS-SAM not only significantly outperforms the recently proposed SAM-Adapter and SSOM, but achieves competitive performance with the SOTA domain-specific models. Our code is available at: https://github.com/maoyangou147/TS-SAM.