mlsanityData to 14 Oct 2026

An Empirical Study on the Robustness of the Segment Anything Model (SAM)

arXiv 2305.06422 ↗First posted 10 May 2023cs.CVcited by 23

Not on the map. Why?

Abstract (arXiv)

The Segment Anything Model (SAM) is a foundation model for general image segmentation. Although it exhibits impressive performance predominantly on natural images, understanding its robustness against various image perturbations and domains is critical for real-world applications where such challenges frequently arise. In this study we conduct a comprehensive robustness investigation of SAM under diverse real-world conditions. Our experiments encompass a wide range of image perturbations. Our experimental results demonstrate that SAM's performance generally declines under perturbed images, with varying degrees of vulnerability across different perturbations. By customizing prompting techniques and leveraging domain knowledge based on the unique characteristics of each dataset, the model's resilience to these perturbations can be enhanced, addressing dataset-specific challenges. This work sheds light on the limitations and strengths of SAM in real-world applications, promoting the development of more robust and versatile image segmentation solutions.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Builds on 26 of its 36 references that are in the map. First posted on arXiv 10 May 2023.

Builds on

26 of 36 references.

References · 26

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  7. 2023Segment Anythingcited by 8,234
  8. 2017Mask R-CNNcited by 7,277
  9. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  10. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  11. 2016Pyramid Scene Parsing Networkcited by 2,479
  12. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
14 more
  1. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  2. 2023Segment Anything Model for Medical Image Analysis: an Experimental Studycited by 291
  3. 2018DeepGlobe 2018: A Challenge to Parse the Earth through Satellite Imagescited by 277
  4. 2021K-Net: Towards Unified Image Segmentationcited by 212
  5. 2023Segment Anything Model (SAM) for Digital Pathology: Assess Zero-shot Segmentation on Whole Slide Imagingcited by 161
  6. 2023Segment Anything Is Not Always Perfect: An Investigation of SAM on Different Real-world Applicationscited by 121
  7. 2019Benchmarking the Robustness of Semantic Segmentation Modelscited by 111
  8. 2017DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentationcited by 110
  9. 2023Can SAM Segment Anything? When SAM Meets Camouflaged Object Detectioncited by 100
  10. 2023SAM Struggles in Concealed Scenes -- Empirical Study on Segment Anythingcited by 75
  11. 2023Polyp-SAM: Transfer SAM for Polyp Segmentationcited by 64
  12. 2023Can SAM Segment Polyps?cited by 59
  13. 2023Are Large Language Models Ready for Healthcare? A Comparative Study on Clinical Language Understandingcited by 40
  14. 2015Instance-Level Segmentation for Autonomous Driving with Deep Densely Connected MRFscited by 32

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 6; 2024: 5; 2025: 9; 2026: 3.

23 papers.

Sort:

By area:

  1. 2023A Comprehensive Survey on Segment Anything Model for Vision and Beyondcited by 64
  2. 2023Metacognitive Prompting Improves Understanding in Large Language Modelscited by 58
  3. 2023A Survey on Segment Anything Model (SAM): Vision Foundation Model Meets Prompt Engineeringcited by 45
  4. 2024RobustSAM: Segment Anything Robustly on Degraded Imagescited by 36
  5. 2023Robustness of SAM: Segment Anything Under Corruptions and Beyondcited by 31
  6. 2023On the Robustness of Segment Anythingcited by 25
  7. 2024Robust Box Prompt based SAM for Medical Image Segmentationcited by 9
  8. 2024Unsegment Anything by Simulating Deformationcited by 6
  9. 2023SAM Meets UAP: Attacking Segment Anything Model With Universal Adversarial Perturbationcited by 4
  10. 2024Red-Teaming Segment Anything Modelcited by 4
  11. 2025SALT: A Flexible Semi-Automatic Labeling Tool for General LiDAR Point Clouds with Cross-Scene Adaptability and 4D Consistencycited by 4
  12. 2025Robust SAM: On the Adversarial Robustness of Vision Foundation Modelscited by 4
11 more
  1. 2025Segment Any-Quality Images with Generative Latent Space Enhancementcited by 3
  2. 2024TETRIS: Towards Exploring the Robustness of Interactive Segmentationcited by 2
  3. 2025RoSe: Robust Self-supervised Stereo Matching under Adverse Weather Conditionscited by 2
  4. 2025The SAM2-to-SAM3 Gap in the Segment Anything Model Family: Why Prompt-Based Expertise Fails in Concept-Driven Image Segmentationcited by 2
  5. 2026BALD-SAM: Disagreement-based Active Prompting in Interactive Segmentationcited by 1
  6. 2025Contour Field based Elliptical Shape Prior for the Segment Anything Modelcited by 0
  7. 2025synth-dacl: Does Synthetic Defect Data Enhance Segmentation Accuracy and Robustness for Real-World Bridge Inspections?cited by 0
  8. 2025Semantic-E2VID: a Semantic-Enriched Paradigm for Event-to-Video Reconstructioncited by 0
  9. 2025Towards Integrating Uncertainty for Domain-Agnostic Segmentationcited by 0
  10. 2026Towards Any-Quality Image Segmentation via Generative and Adaptive Latent Space Enhancementcited by 0
  11. 2026BREPS: Bounding-Box Robustness Evaluation of Promptable Segmentationcited by 0