# TextureSAM: Towards a Texture Aware Foundation Model for Segmentation

arXiv: https://arxiv.org/abs/2505.16540 · HTML: https://mlsanity.com/paper/2505.16540/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 29 references that are in the map.
- First posted on arXiv 22 May 2025.

## Builds on (most cited first, 12 of 12)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Intriguing Properties of Vision Transformers](https://mlsanity.com/paper/2105.10497/) (2021)
- [Segment Anything Is Not Always Perfect: An Investigation of SAM on Different Real-world Applications](https://mlsanity.com/paper/2304.05750/) (2023)
- [Fast Edge Detection Using Structured Forests](https://mlsanity.com/paper/1406.5549/) (2014)
- [DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection](https://mlsanity.com/paper/1412.1123/) (2014)
- [Pushing the Boundaries of Boundary Detection using Deep Learning](https://mlsanity.com/paper/1511.07386/) (2015)
- [Compositional Neural Textures](https://mlsanity.com/paper/2404.12509/) (2024)
- [An End-to-End Computer Vision Methodology for Quantitative Metallography](https://mlsanity.com/paper/2104.11159/) (2021)

## Built on by (most cited first, 3 of 3)

- [Sub-Semantic Image Segmentation](https://mlsanity.com/paper/2606.14754/) (2026)
- [Where Does Texture Evidence Live in SAM? Features, Proposal Masks, and Texture Segmentation](https://mlsanity.com/paper/2606.14755/) (2026)
- [Texture-Shape Bias Balancing for Robust Synthetic-to-Real Semantic Segmentation in Automotive NIR Imagery](https://mlsanity.com/paper/2606.15072/) (2026)

## Abstract (arXiv)

Segment Anything Models (SAM) have achieved remarkable success in object segmentation tasks across diverse datasets. However, these models are predominantly trained on large-scale semantic segmentation datasets, which introduce a bias toward object shape rather than texture cues in the image. This limitation is critical in domains such as medical imaging, material classification, and remote sensing, where texture changes define object boundaries. In this study, we investigate SAM's bias toward semantics over textures and introduce a new texture-aware foundation model, TextureSAM, which performs superior segmentation in texture-dominant scenarios. To achieve this, we employ a novel fine-tuning approach that incorporates texture augmentation techniques, incrementally modifying training images to emphasize texture features. By leveraging a novel texture-alternation of the ADE20K dataset, we guide TextureSAM to prioritize texture-defined regions, thereby mitigating the inherent shape bias present in the original SAM model. Our extensive experiments demonstrate that TextureSAM significantly outperforms SAM-2 on both natural (+0.2 mIoU) and synthetic (+0.18 mIoU) texture-based segmentation datasets. The code and texture-augmented dataset will be publicly available.