# Robustness of SAM: Segment Anything Under Corruptions and Beyond

arXiv: https://arxiv.org/abs/2306.07713 · HTML: https://mlsanity.com/paper/2306.07713/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 81 of its 119 references that are in the map.
- First posted on arXiv 13 Jun 2023.

## Builds on (most cited first, 30 of 81)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples](https://mlsanity.com/paper/1802.00420/) (2018)
- [Adversarial Machine Learning at Scale](https://mlsanity.com/paper/1611.01236/) (2016)
- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [Florence: A New Foundation Model for Computer Vision](https://mlsanity.com/paper/2111.11432/) (2021)
- [Adversarial Patch](https://mlsanity.com/paper/1712.09665/) (2017)

## Built on by (most cited first, 30 of 31)

- [MobileSAMv2: Faster Segment Anything to Everything](https://mlsanity.com/paper/2312.09579/) (2023)
- [RobustSAM: Segment Anything Robustly on Degraded Images](https://mlsanity.com/paper/2406.09627/) (2024)
- [Robustness of Segment Anything Model (SAM) for Autonomous Driving in Adverse Weather Conditions](https://mlsanity.com/paper/2306.13290/) (2023)
- [Black-box Targeted Adversarial Attack on Segment Anything (SAM)](https://mlsanity.com/paper/2310.10010/) (2023)
- [SamDSK: Combining Segment Anything Model with Domain-Specific Knowledge for Semi-Supervised Learning in Medical Image Segmentation](https://mlsanity.com/paper/2308.13759/) (2023)
- [Practical Region-level Attack against Segment Anything Models](https://mlsanity.com/paper/2404.08255/) (2024)
- [Robust Box Prompt based SAM for Medical Image Segmentation](https://mlsanity.com/paper/2407.21284/) (2024)
- [Internal-External Boundary Attention Fusion for Glass Surface Segmentation](https://mlsanity.com/paper/2307.00212/) (2023)
- [On the Out of Distribution Robustness of Foundation Models in Medical Image Segmentation](https://mlsanity.com/paper/2311.11096/) (2023)
- [Benchmarking PathCLIP for Pathology Image Analysis](https://mlsanity.com/paper/2401.02651/) (2024)
- [Unsegment Anything by Simulating Deformation](https://mlsanity.com/paper/2404.02585/) (2024)
- [Segment (Almost) Nothing: Prompt-Agnostic Adversarial Attacks on Segmentation Models](https://mlsanity.com/paper/2311.14450/) (2023)
- [Segment-Anything Models Achieve Zero-shot Robustness in Autonomous Driving](https://mlsanity.com/paper/2408.09839/) (2024)
- [Boosting Federated Learning Convergence with Prototype Regularization](https://mlsanity.com/paper/2307.10575/) (2023)
- [SAM Meets UAP: Attacking Segment Anything Model With Universal Adversarial Perturbation](https://mlsanity.com/paper/2310.12431/) (2023)
- [Red-Teaming Segment Anything Model](https://mlsanity.com/paper/2404.02067/) (2024)
- [Logit Calibration and Feature Contrast for Robust Federated Learning on Non-IID Data](https://mlsanity.com/paper/2404.06776/) (2024)
- [Learning from Unlabelled Data with Transformers: Domain Adaptation for Semantic Segmentation of High Resolution Aerial Images](https://mlsanity.com/paper/2404.11299/) (2024)
- [Robust SAM: On the Adversarial Robustness of Vision Foundation Models](https://mlsanity.com/paper/2504.08906/) (2025)
- [Towards Dependable Optical Flow: Benchmarks for Real-World Spatiotemporal Corruptions](https://mlsanity.com/paper/2411.14865/) (2024)
- [Curriculum Fine-tuning of Vision Foundation Model for Medical Image Classification Under Label Noise](https://mlsanity.com/paper/2412.00150/) (2024)
- [Segment Any-Quality Images with Generative Latent Space Enhancement](https://mlsanity.com/paper/2503.12507/) (2025)
- [TETRIS: Towards Exploring the Robustness of Interactive Segmentation](https://mlsanity.com/paper/2402.06132/) (2024)
- [RoSe: Robust Self-supervised Stereo Matching under Adverse Weather Conditions](https://mlsanity.com/paper/2509.19165/) (2025)
- [BoxCell: Leveraging SAM for Cell Segmentation with Box Supervision](https://mlsanity.com/paper/2311.17960/) (2023)
- [Towards Communication-Efficient Adversarial Federated Learning for Robust Edge Intelligence](https://mlsanity.com/paper/2501.15257/) (2025)
- [Towards Any-Quality Image Segmentation via Generative and Adaptive Latent Space Enhancement](https://mlsanity.com/paper/2601.02018/) (2026)
- [DeCoDrift: Stabilizing Decoder Coupling in Closed-Loop Foundation Segmentation](https://mlsanity.com/paper/2605.25730/) (2026)
- [PGE-SAM: Prompt-Guided Feature Enhancement for Interactive Segmentation under Degradation](https://mlsanity.com/paper/2606.30477/) (2026)
- [Universal Concept Disruption for SAM3 Image Segmentation](https://mlsanity.com/paper/2608.05983/) (2026)

## Abstract (arXiv)

Segment anything model (SAM), as the name suggests, is claimed to be capable of cutting out any object and demonstrates impressive zero-shot transfer performance with the guidance of prompts. However, there is currently a lack of comprehensive evaluation regarding its robustness under various corruptions. Understanding the robustness of SAM across different corruption scenarios is crucial for its real-world deployment. Prior works show that SAM is biased towards texture (style) rather than shape, motivated by which we start by investigating its robustness against style transfer, which is synthetic corruption. Following by interpreting the effects of synthetic corruption as style changes, we proceed to conduct a comprehensive evaluation for its robustness against 15 types of common corruption. These corruptions mainly fall into categories such as digital, noise, weather, and blur, and within each corruption category, we explore 5 severity levels to simulate real-world corruption scenarios. Beyond the corruptions, we further assess the robustness of SAM against local occlusion and local adversarial patch attacks. To the best of our knowledge, our work is the first of its kind to evaluate the robustness of SAM under style change, local occlusion, and local adversarial patch attacks. Given that patch attacks visible to human eyes are easily detectable, we further assess its robustness against global adversarial attacks that are imperceptible to human eyes. Overall, this work provides a comprehensive empirical study of the robustness of SAM, evaluating its performance under various corruptions and extending the assessment to critical aspects such as local occlusion, local adversarial patch attacks, and global adversarial attacks. These evaluations yield valuable insights into the practical applicability and effectiveness of SAM in addressing real-world challenges.