# Multi-Scale and Detail-Enhanced Segment Anything Model for Salient Object Detection

arXiv: https://arxiv.org/abs/2408.04326 · HTML: https://mlsanity.com/paper/2408.04326/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 67 references that are in the map.
- First posted on arXiv 8 Aug 2024.

## Builds on (most cited first, 30 of 40)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks](https://mlsanity.com/paper/2401.14159/) (2024)
- [Structure-measure: A New Way to Evaluate Foreground Maps](https://mlsanity.com/paper/1708.00786/) (2017)
- [EGNet:Edge Guidance Network for Salient Object Detection](https://mlsanity.com/paper/1908.08297/) (2019)
- [Anabranch Network for Camouflaged Object Segmentation](https://mlsanity.com/paper/2105.09451/) (2021)
- [Segment Anything in Medical Images](https://mlsanity.com/paper/2304.12306/) (2023)
- [Cascaded Partial Decoder for Fast and Accurate Salient Object Detection](https://mlsanity.com/paper/1904.08739/) (2019)
- [A Simple Pooling-Based Design for Real-Time Salient Object Detection](https://mlsanity.com/paper/1904.09569/) (2019)
- [Multi-scale Interactive Network for Salient Object Detection](https://mlsanity.com/paper/2007.09062/) (2020)
- [Concealed Object Detection](https://mlsanity.com/paper/2102.10274/) (2021)
- [The Secrets of Salient Object Segmentation](https://mlsanity.com/paper/1406.2807/) (2014)
- [F3Net: Fusion, Feedback and Focus for Salient Object Detection](https://mlsanity.com/paper/1911.11445/) (2019)
- [Visual Saliency Transformer](https://mlsanity.com/paper/2104.12099/) (2021)
- [Polyp-PVT: Polyp Segmentation with Pyramid Vision Transformers](https://mlsanity.com/paper/2108.06932/) (2021)
- [Salient Object Detection via Integrity Learning](https://mlsanity.com/paper/2101.07663/) (2021)
- [Label Decoupling Framework for Salient Object Detection](https://mlsanity.com/paper/2008.11048/) (2020)
- [Amulet: Aggregating Multi-level Convolutional Features for Salient Object Detection](https://mlsanity.com/paper/1708.02001/) (2017)
- [Towards High-Resolution Salient Object Detection](https://mlsanity.com/paper/1908.07274/) (2019)
- [Context-aware Cross-level Fusion Network for Camouflaged Object Detection](https://mlsanity.com/paper/2105.12555/) (2021)
- [Selectivity or Invariance: Boundary-aware Salient Object Detection](https://mlsanity.com/paper/1812.10066/) (2018)
- [Learning Uncertain Convolutional Features for Accurate Saliency Detection](https://mlsanity.com/paper/1708.02031/) (2017)

## Built on by (most cited first, 30 of 30)

- [SAM2-UNet: Segment Anything 2 Makes Strong Encoder for Natural and Medical Image Segmentation](https://mlsanity.com/paper/2408.08870/) (2024)
- [SAM-COD: SAM-guided Unified Framework for Weakly-Supervised Camouflaged Object Detection](https://mlsanity.com/paper/2408.10760/) (2024)
- [Controllable-LPMoE: Adapting to Challenging Object Segmentation via Dynamic Local Priors from Mixture-of-Experts](https://mlsanity.com/paper/2510.21114/) (2025)
- [Rethinking Detecting Salient and Camouflaged Objects in Unconstrained Scenes](https://mlsanity.com/paper/2412.10943/) (2024)
- [FSDENet: A Frequency and Spatial Domains based Detail Enhancement Network for Remote Sensing Semantic Segmentation](https://mlsanity.com/paper/2510.00059/) (2025)
- [Evaluation Study on SAM 2 for Class-agnostic Instance-level Segmentation](https://mlsanity.com/paper/2409.02567/) (2024)
- [KAN-SAM: Kolmogorov-Arnold Network Guided Segment Anything Model for RGB-T Salient Object Detection](https://mlsanity.com/paper/2504.05878/) (2025)
- [Zero-Shot Subject-Centric Generation for Creative Application Using Entropy Fusion](https://mlsanity.com/paper/2503.10697/) (2025)
- [SAMSOD: Rethinking SAM Optimization for RGB-T Salient Object Detection](https://mlsanity.com/paper/2510.03689/) (2025)
- [HyPSAM: Hybrid Prompt-driven Segment Anything Model for RGB-Thermal Salient Object Detection](https://mlsanity.com/paper/2509.18738/) (2025)
- [HALO: Human-Aligned End-to-end Image Retargeting with Layered Transformations](https://mlsanity.com/paper/2504.03026/) (2025)
- [DeshadowMamba: Deshadowing as 1D Sequential Similarity](https://mlsanity.com/paper/2510.24260/) (2025)
- [LFX: Towards Unified Light Field Dense Semantic Segmentation and Salient Object Detection](https://mlsanity.com/paper/2503.00747/) (2025)
- [LSP-ST: Ladder Shape-Biased Side-Tuning for Robust Infrared Small Target Detection](https://mlsanity.com/paper/2504.14481/) (2025)
- [Unleashing the Power of Motion and Depth: A Selective Fusion Strategy for RGB-D Video Salient Object Detection](https://mlsanity.com/paper/2507.21857/) (2025)
- [SPLF-SAM: Self-Prompting Segment Anything Model for Light Field Salient Object Detection](https://mlsanity.com/paper/2508.19746/) (2025)
- [RDTE-UNet: A Boundary and Detail Aware UNet for Precise Medical Image Segmentation](https://mlsanity.com/paper/2511.01328/) (2025)
- [DualGazeNet: A Biologically Inspired Dual-Gaze Query Network for Salient Object Detection](https://mlsanity.com/paper/2511.18865/) (2025)
- [A Unified and Controllable Framework for Layered Image Generation with Visual Effects](https://mlsanity.com/paper/2601.15507/) (2026)
- [Revisiting Salient Object Detection from an Observer-Centric Perspective](https://mlsanity.com/paper/2602.06369/) (2026)
- [SAMSEM -- A Generic and Scalable Approach for IC Metal Line Segmentation](https://mlsanity.com/paper/2603.16548/) (2026)
- [Language-Guided Structure-Aware Network for Camouflaged Object Detection](https://mlsanity.com/paper/2603.24355/) (2026)
- [IP-SAM: Rethinking Prompt-Conditioned Segmentation for Prompt-Absent Deployment](https://mlsanity.com/paper/2603.27250/) (2026)
- [UniV2D: Bridging Visual Restoration and Semantic Perception for Underwater Salient Object Detection](https://mlsanity.com/paper/2605.07146/) (2026)
- [M4-SAM: Multi-Modal Mixture-of-Experts with Memory-Augmented SAM for RGB-D Video Salient Object Detection](https://mlsanity.com/paper/2605.11760/) (2026)
- [Decoupled Motion Representation Learning for Moving Infrared Small Target Detection](https://mlsanity.com/paper/2606.15286/) (2026)
- [Rethinking Conditional Generation for Underwater Salient Object Detection](https://mlsanity.com/paper/2607.01825/) (2026)
- [Spectral-Spatial Synergistic Guided Network for Hyperspectral Salient Object Detection](https://mlsanity.com/paper/2607.21032/) (2026)
- [RDVSv2: A Large-scale Benchmark for RGB-D Video Salient Object Detection](https://mlsanity.com/paper/2607.25392/) (2026)
- [AttenIR: Aligning Dense Visual Attention with Mesh Warping for Unsupervised Image Retargeting](https://mlsanity.com/paper/2610.17743/) (2026)

## Abstract (arXiv)

Salient Object Detection (SOD) aims to identify and segment the most prominent objects in images. Advanced SOD methods often utilize various Convolutional Neural Networks (CNN) or Transformers for deep feature extraction. However, these methods still deliver low performance and poor generalization in complex cases. Recently, Segment Anything Model (SAM) has been proposed as a visual fundamental model, which gives strong segmentation and generalization capabilities. Nonetheless, SAM requires accurate prompts of target objects, which are unavailable in SOD. Additionally, SAM lacks the utilization of multi-scale and multi-level information, as well as the incorporation of fine-grained details. To address these shortcomings, we propose a Multi-scale and Detail-enhanced SAM (MDSAM) for SOD. Specifically, we first introduce a Lightweight Multi-Scale Adapter (LMSA), which allows SAM to learn multi-scale information with very few trainable parameters. Then, we propose a Multi-Level Fusion Module (MLFM) to comprehensively utilize the multi-level information from the SAM's encoder. Finally, we propose a Detail Enhancement Module (DEM) to incorporate SAM with fine-grained details. Experimental results demonstrate the superior performance of our model on multiple SOD datasets and its strong generalization on other segmentation tasks. The source code is released at https://github.com/BellyBeauty/MDSAM.