mlsanityData to 7 Oct 2026

DarkSAM: Fooling Segment Anything Model to Segment Nothing

arXiv 2409.17874 ↗First posted 26 Sep 2024cs.AIcited by 32

Not on the map. Why?

Abstract (arXiv)

Segment Anything Model (SAM) has recently gained much attention for its outstanding generalization to unseen data and tasks. Despite its promising prospect, the vulnerabilities of SAM, especially to universal adversarial perturbation (UAP) have not been thoroughly investigated yet. In this paper, we propose DarkSAM, the first prompt-free universal attack framework against SAM, including a semantic decoupling-based spatial attack and a texture distortion-based frequency attack. We first divide the output of SAM into foreground and background. Then, we design a shadow target strategy to obtain the semantic blueprint of the image as the attack target. DarkSAM is dedicated to fooling SAM by extracting and destroying crucial object features from images in both spatial and frequency domains. In the spatial domain, we disrupt the semantics of both the foreground and background in the image to confuse SAM. In the frequency domain, we further enhance the attack effectiveness by distorting the high-frequency components (i.e., texture information) of the image. Consequently, with a single UAP, DarkSAM renders SAM incapable of segmenting objects across diverse images with varying prompts. Experimental results on four datasets for SAM and its two variant models demonstrate the powerful attack capability and transferability of DarkSAM.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 38 of its 45 references that are in the map. First posted on arXiv 26 Sep 2024.

Builds on

38 of 45 references.

References · 38

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2023Segment Anythingcited by 8,158
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  6. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  7. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  8. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  9. 2016Pyramid Scene Parsing Networkcited by 2,472
  10. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  11. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
  12. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
26 more
  1. 2016Universal adversarial perturbationscited by 774
  2. 2023Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentationcited by 485
  3. 2023Segment Anything in High Qualitycited by 373
  4. 2023Faster Segment Anything: Towards Lightweight SAM for Mobile Applicationscited by 345
  5. 2023Personalize Segment Anything Model with One Shotcited by 250
  6. 2023Track Anything: Segment Anything Meets Videoscited by 230
  7. 2023Semantic-SAM: Segment and Recognize Anything at Any Granularitycited by 185
  8. 2020A Self-supervised Approach for Adversarial Robustnesscited by 180
  9. 2017On the Robustness of Semantic Segmentation Models to Adversarial Attackscited by 126
  10. 2023AdvCLIP: Downstream-agnostic Adversarial Examples in Multimodal Contrastive Learningcited by 101
  11. 2017Universal Adversarial Perturbations Against Semantic Image Segmentationcited by 92
  12. 2018Generalizable Data-free Objective for Crafting Universal Adversarial Perturbationscited by 88
  13. 2023MA-SAM: Modality-agnostic SAM Adaptation for 3D Medical Image Segmentationcited by 82
  14. 2022SegPGD: An Effective and Efficient Adversarial Attack for Evaluating and Boosting Segmentation Robustnesscited by 74
  15. 2017Fast Feature Fool: A data independent approach to universal adversarial perturbationscited by 65
  16. 2023Polyp-SAM: Transfer SAM for Polyp Segmentationcited by 64
  17. 2017NAG: Network for Adversary Generationcited by 51
  18. 2023SAM-guided Graph Cut for 3D Instance Segmentationcited by 35
  19. 2023Why Does Little Robustness Help? A Further Step Towards Understanding Adversarial Transferabilitycited by 32
  20. 2023Downstream-agnostic Adversarial Examplescited by 32
  21. 2018Ask, Acquire, and Attack: Data-free UAP Generation using Class Impressionscited by 31
  22. 2024Transferable Adversarial Facial Images for Privacy Protectioncited by 26
  23. 2023On the Robustness of Segment Anythingcited by 24
  24. 2024Securely Fine-tuning Pre-trained Encoders Against Adversarial Examplescited by 21
  25. 2023Understanding Segment Anything Model: SAM is Biased Towards Texture Rather than Shapecited by 10
  26. 2023Segment (Almost) Nothing: Prompt-Agnostic Adversarial Attacks on Segmentation Modelscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 16; 2026: 10.

32 papers.

Sort:

By area:

  1. 2025X-Transfer Attacks: Towards Super Transferable Adversarial Attacks on CLIPcited by 21
  2. 2024Transferable Adversarial Attacks on SAM and Its Downstream Modelscited by 20
  3. 2025ADVEDM:Fine-grained Adversarial Attack against VLM-based Embodied Agentscited by 19
  4. 2024Poison-splat: Computation Cost Attack on 3D Gaussian Splattingcited by 17
  5. 2024NumbOD: A Spatial-Frequency Fusion Attack Against Object Detectorscited by 17
  6. 2024Unlearnable 3D Point Clouds: Class-wise Transformation Is All You Needcited by 16
  7. 2024PB-UAP: Hybrid Universal Adversarial Attack For Image Segmentationcited by 14
  8. 2025Vanish into Thin Air: Cross-prompt Universal Adversarial Attacks for SAM2cited by 11
  9. 2025When Robots Obey the Patch: Universal Transferable Patch Attacks on Vision-Language-Action Modelscited by 9
  10. 2025Towards Reliable Forgetting: A Survey on Machine Unlearning Verificationcited by 8
  11. 2025SegTrans: Transferable Adversarial Examples for Segmentation Modelscited by 8
  12. 2025ForensicsSAM: Toward Robust and Unified Image Forgery Detection and Localization Resisting to Adversarial Attackcited by 6
20 more
  1. 2025DarkHash: A Data-Free Backdoor Attack Against Deep Hashingcited by 6
  2. 2026Time Is All It Takes: Spike-Retiming Attacks on Event-Driven Spiking Neural Networkscited by 5
  3. 2025Mix-QSAM: Mixed-Precision Quantization of the Segment Anything Modelcited by 4
  4. 2025Improving Generalization of Universal Adversarial Perturbation via Dynamic Maximin Optimizationcited by 3
  5. 2026Erosion Attack for Adversarial Training to Enhance Semantic Segmentation Robustnesscited by 3
  6. 2024Region-Guided Attack on the Segment Anything Model (SAM)cited by 2
  7. 2025UFVideo: Towards Unified Fine-Grained Video Cooperative Understanding with Large Language Modelscited by 2
  8. 2025Towards Real-World Deepfake Detection: A Diverse In-the-wild Dataset of Forgery Facescited by 1
  9. 2025Dual-View Inference Attack: Machine Unlearning Amplifies Privacy Exposurecited by 1
  10. 2026Transferable Physical-World Adversarial Patches Against Object Detection in Autonomous Drivingcited by 1
  11. 2025LSP-ST: Ladder Shape-Biased Side-Tuning for Robust Infrared Small Target Detectioncited by 0
  12. 2025SAM Encoder Breach by Adversarial Simplicial Complex Triggers Downstream Model Failurescited by 0
  13. 2025Malicious Image Analysis via Vision-Language Segmentation Fusion: Detection, Element, and Location in One-shotcited by 0
  14. 2026Low-Cost Hard-Label Adversarial Attack with Theoretical Foundationscited by 0
  15. 2026BREPS: Bounding-Box Robustness Evaluation of Promptable Segmentationcited by 0
  16. 2026DarkLLM: Learning Language-Driven Adversarial Attacks with Large Language Modelscited by 0
  17. 2026VFACamou: View-Fused Adversarial Camouflage for Environment-Adaptive Physical Evasioncited by 0
  18. 2026Universal Concept Disruption for SAM3 Image Segmentationcited by 0
  19. 2026Generating Multi-view Adversarial Examples for Visual Geometry Grounded Transformercited by 0
  20. 2026Universal Cross-Prompt Adversarial Attacks on Promptable Concept Segmentationcited by 0