mlsanityData to 7 Oct 2026

Understanding Segment Anything Model: SAM is Biased Towards Texture Rather than Shape

arXiv 2311.11465 ↗First posted 3 Jun 2023cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

In contrast to the human vision that mainly depends on the shape for recognizing the objects, deep image recognition models are widely known to be biased toward texture. Recently, Meta research team has released the first foundation model for image segmentation, termed segment anything model (SAM), which has attracted significant attention. In this work, we understand SAM from the perspective of texture textitv.s. shape. Different from label-oriented recognition tasks, the SAM is trained to predict a mask for covering the object shape based on a promt. With this said, it seems self-evident that the SAM is biased towards shape. In this work, however, we reveal an interesting finding: the SAM is strongly biased towards texture-like dense features rather than shape. This intriguing finding is supported by a novel setup where we disentangle texture and shape cues and design texture-shape cue conflict for mask prediction.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 34 of its 48 references that are in the map. First posted on arXiv 3 Jun 2023.

Builds on

34 of 48 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2023Segment Anythingcited by 8,158
  7. 2016Densely Connected Convolutional Networkscited by 6,691
  8. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  9. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  10. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  11. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  12. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
22 more
  1. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  2. 2021Florence: A New Foundation Model for Computer Visioncited by 802
  3. 2021Intriguing Properties of Vision Transformerscited by 446
  4. 2023Text-to-image Diffusion Models in Generative AI: A Surveycited by 264
  5. 2019Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNetcited by 260
  6. 2023Inpaint Anything: Segment Anything Meets Image Inpaintingcited by 233
  7. 2023Track Anything: Segment Anything Meets Videoscited by 230
  8. 2023Can SAM Segment Anything? When SAM Meets Camouflaged Object Detectioncited by 99
  9. 2023Anything-3D: Towards Single-view Anything Reconstruction in the Wildcited by 91
  10. 2023A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?cited by 84
  11. 2023A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AIcited by 68
  12. 2023Generative AI meets 3D: A Survey on Text-to-3D in AIGC Eracited by 61
  13. 2022A Survey on Masked Autoencoder for Self-supervised Learning in Vision and Beyondcited by 58
  14. 2023Input Augmentation with SAM: Boosting Medical Image Segmentation with Segmentation Foundation Modelcited by 56
  15. 2023One Small Step for Generative AI, One Giant Leap for AGI: A Complete Survey on ChatGPT in AIGC Eracited by 55
  16. 2017Cognitive Psychology for Deep Neural Networks: A Shape Bias Case Studycited by 46
  17. 2023A Survey on Segment Anything Model (SAM): Vision Foundation Model Meets Prompt Engineeringcited by 45
  18. 2022Grad-StyleSpeech: Any-speaker Adaptive Text-to-Speech Synthesis with Diffusion Modelscited by 34
  19. 2023Segment Anything Model (SAM) Meets Glass: Mirror and Transparent Objects Cannot Be Easily Detectedcited by 34
  20. 2018Assessing Shape Bias Property of Convolutional Neural Networkscited by 18
  21. 2023Segment Anything in Non-Euclidean Domains: Challenges and Opportunitiescited by 16
  22. 2020ResNet or DenseNet? Introducing Dense Shortcuts to ResNetcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 3; 2025: 3; 2026: 1.

10 papers.

  1. 2024DarkSAM: Fooling Segment Anything Model to Segment Nothingcited by 32
  2. 2023Robustness of SAM: Segment Anything Under Corruptions and Beyondcited by 30
  3. 2023Internal-External Boundary Attention Fusion for Glass Surface Segmentationcited by 6
  4. 2024Quantifying the Limits of Segmentation Foundation Models: Modeling Challenges in Segmenting Tree-Like and Low-Contrast Objectscited by 5
  5. 2023SAM Meets UAP: Attacking Segment Anything Model With Universal Adversarial Perturbationcited by 4
  6. 2024Red-Teaming Segment Anything Modelcited by 4
  7. 2025Transferring Styles for Reduced Texture Bias and Improved Robustness in Semantic Segmentation Networkscited by 1
  8. 2026Physical Object Understanding with a Physically Controllable World Modelcited by 1
  9. 2025Contour Field based Elliptical Shape Prior for the Segment Anything Modelcited by 0
  10. 2025LSP-ST: Ladder Shape-Biased Side-Tuning for Robust Infrared Small Target Detectioncited by 0