# Segment Anything Is Not Always Perfect: An Investigation of SAM on Different Real-world Applications

arXiv: https://arxiv.org/abs/2304.05750 · HTML: https://mlsanity.com/paper/2304.05750/

## Key facts

- Cited by 121 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 66 references that are in the map.
- First posted on arXiv 12 Apr 2023.

## Builds on (most cited first, 27 of 27)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)](https://mlsanity.com/paper/1710.05006/) (2017)
- [PraNet: Parallel Reverse Attention Network for Polyp Segmentation](https://mlsanity.com/paper/2006.11392/) (2020)
- [Concealed Object Detection](https://mlsanity.com/paper/2102.10274/) (2021)
- [Visual Saliency Transformer](https://mlsanity.com/paper/2104.12099/) (2021)
- [Camouflaged Object Segmentation with Distraction Mining](https://mlsanity.com/paper/2104.10475/) (2021)
- [Zoom In and Out: A Mixed-scale Triplet Network for Camouflaged Object Detection](https://mlsanity.com/paper/2203.02688/) (2022)
- [Salient Object Detection via Integrity Learning](https://mlsanity.com/paper/2101.07663/) (2021)
- [Suppress and Balance: A Simple Gated Network for Salient Object Detection](https://mlsanity.com/paper/2007.08074/) (2020)
- [Highly Accurate Dichotomous Image Segmentation](https://mlsanity.com/paper/2203.03041/) (2022)
- [Can SAM Segment Anything? When SAM Meets Camouflaged Object Detection](https://mlsanity.com/paper/2304.04709/) (2023)
- [Salient Object Detection: A Survey](https://mlsanity.com/paper/1411.5878/) (2014)
- [Video Polyp Segmentation: A Deep Learning Perspective](https://mlsanity.com/paper/2203.14291/) (2022)
- [High-resolution Iterative Feedback Network for Camouflaged Object Detection](https://mlsanity.com/paper/2203.11624/) (2022)
- [Joint Optic Disc and Cup Segmentation Based on Multi-label Deep Network and Polar Transformation](https://mlsanity.com/paper/1801.00926/) (2018)
- [Deep Gradient Learning for Efficient Camouflaged Object Detection](https://mlsanity.com/paper/2205.12853/) (2022)
- [EDN: Salient Object Detection via Extremely-Downsampled Network](https://mlsanity.com/paper/2012.13093/) (2020)
- [RGB-T Image Saliency Detection via Collaborative Graph Learning](https://mlsanity.com/paper/1905.06741/) (2019)
- [Direction-aware Spatial Context Features for Shadow Detection](https://mlsanity.com/paper/1712.04142/) (2017)
- [RGB-D Saliency Detection via Cascaded Mutual Information Minimization](https://mlsanity.com/paper/2109.07246/) (2021)
- [Feature Aggregation and Propagation Network for Camouflaged Object Detection](https://mlsanity.com/paper/2212.00990/) (2022)
- [Learning Selective Mutual Attention and Contrast for RGB-D Saliency Detection](https://mlsanity.com/paper/2010.05537/) (2020)
- [Revisiting Shadow Detection: A New Benchmark Dataset for Complex World](https://mlsanity.com/paper/1911.06998/) (2019)
- [Promoting Saliency From Depth: Deep Unsupervised RGB-D Saliency Detection](https://mlsanity.com/paper/2205.07179/) (2022)
- [BlazeNeo: Blazing fast polyp segmentation and neoplasm detection](https://mlsanity.com/paper/2203.00129/) (2022)
- [Label-efficient Hybrid-supervised Learning for Medical Image Segmentation](https://mlsanity.com/paper/2203.05956/) (2022)

## Built on by (most cited first, 30 of 121)

- [Segment Anything in Medical Images](https://mlsanity.com/paper/2304.12306/) (2023)
- [SAM-6D: Segment Anything Model Meets Zero-Shot 6D Object Pose Estimation](https://mlsanity.com/paper/2311.15707/) (2023)
- [Segment Anything Model for Medical Image Segmentation: Current Applications and Future Directions](https://mlsanity.com/paper/2401.03495/) (2024)
- [Weakly-Supervised Concealed Object Segmentation with SAM-based Pseudo Labeling and Multi-scale Feature Grouping](https://mlsanity.com/paper/2305.11003/) (2023)
- [SAM on Medical Images: A Comprehensive Study on Three Prompt Modes](https://mlsanity.com/paper/2305.00035/) (2023)
- [Review of Large Vision Models and Visual Prompt Engineering](https://mlsanity.com/paper/2307.00855/) (2023)
- [What Matters When Repurposing Diffusion Models for General Dense Perception Tasks?](https://mlsanity.com/paper/2403.06090/) (2024)
- [3DSAM-adapter: Holistic adaptation of SAM from 2D to 3D for promptable tumor segmentation](https://mlsanity.com/paper/2306.13465/) (2023)
- [Convolution Meets LoRA: Parameter Efficient Finetuning for Segment Anything Model](https://mlsanity.com/paper/2401.17868/) (2024)
- [A Comprehensive Survey on Segment Anything Model for Vision and Beyond](https://mlsanity.com/paper/2305.08196/) (2023)
- [Can SAM Segment Polyps?](https://mlsanity.com/paper/2304.07583/) (2023)
- [Med-UniC: Unifying Cross-Lingual Medical Vision-Language Pre-Training by Diminishing Bias](https://mlsanity.com/paper/2305.19894/) (2023)
- [Adapting Segment Anything Model for Change Detection in HR Remote Sensing Images](https://mlsanity.com/paper/2309.01429/) (2023)
- [SayNav: Grounding Large Language Models for Dynamic Planning to Navigation in New Environments](https://mlsanity.com/paper/2309.04077/) (2023)
- [Generalist Vision Foundation Models for Medical Imaging: A Case Study of Segment Anything Model on Zero-Shot Medical Segmentation](https://mlsanity.com/paper/2304.12637/) (2023)
- [Annotation-free Audio-Visual Segmentation](https://mlsanity.com/paper/2305.11019/) (2023)
- [A Survey on Segment Anything Model (SAM): Vision Foundation Model Meets Prompt Engineering](https://mlsanity.com/paper/2306.06211/) (2023)
- [Segment anything, from space?](https://mlsanity.com/paper/2304.13000/) (2023)
- [SAM3D: Zero-Shot 3D Object Detection via Segment Anything Model](https://mlsanity.com/paper/2306.02245/) (2023)
- [Segment Anything Model for Medical Images?](https://mlsanity.com/paper/2304.14660/) (2023)
- [SAMAug: Point Prompt Augmentation for Segment Anything Model](https://mlsanity.com/paper/2307.01187/) (2023)
- [A Survey of Camouflaged Object Detection and Beyond](https://mlsanity.com/paper/2408.14562/) (2024)
- [RobustSAM: Segment Anything Robustly on Degraded Images](https://mlsanity.com/paper/2406.09627/) (2024)
- [Self-Prompting Large Vision Models for Few-Shot Medical Image Segmentation](https://mlsanity.com/paper/2308.07624/) (2023)
- [DFormerv2: Geometry Self-Attention for RGBD Semantic Segmentation](https://mlsanity.com/paper/2504.04701/) (2025)
- [Foundation Models for Biomedical Image Segmentation: A Survey](https://mlsanity.com/paper/2401.07654/) (2024)
- [CAT-SAM: Conditional Tuning for Few-Shot Adaptation of Segment Anything Model](https://mlsanity.com/paper/2402.03631/) (2024)
- [On the Robustness of Segment Anything](https://mlsanity.com/paper/2305.16220/) (2023)
- [Segment Anything Model (SAM) for Radiation Oncology](https://mlsanity.com/paper/2306.11730/) (2023)
- [An Empirical Study on the Robustness of the Segment Anything Model (SAM)](https://mlsanity.com/paper/2305.06422/) (2023)

## Abstract (arXiv)

Recently, Meta AI Research approaches a general, promptable Segment Anything Model (SAM) pre-trained on an unprecedentedly large segmentation dataset (SA-1B). Without a doubt, the emergence of SAM will yield significant benefits for a wide array of practical image segmentation applications. In this study, we conduct a series of intriguing investigations into the performance of SAM across various applications, particularly in the fields of natural images, agriculture, manufacturing, remote sensing, and healthcare. We analyze and discuss the benefits and limitations of SAM, while also presenting an outlook on its future development in segmentation tasks. By doing so, we aim to give a comprehensive understanding of SAM's practical applications. This work is expected to provide insights that facilitate future research activities toward generic segmentation. Source code is publicly available.