mlsanityData to 7 Oct 2026

A Comparative Survey of Vision Transformers for Feature Extraction in Texture Analysis

arXiv 2406.06136 ↗First posted 10 Jun 2024cs.CV cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Texture, a significant visual attribute in images, has been extensively investigated across various image recognition applications. Convolutional Neural Networks (CNNs), which have been successful in many computer vision tasks, are currently among the best texture analysis approaches. On the other hand, Vision Transformers (ViTs) have been surpassing the performance of CNNs on tasks such as object recognition, causing a paradigm shift in the field. However, ViTs have so far not been scrutinized for texture recognition, hindering a proper appreciation of their potential in this specific setting. For this reason, this work explores various pre-trained ViT architectures when transferred to tasks that rely on textures. We review 21 different ViT variants and perform an extensive evaluation and comparison with CNNs and hand-engineered models on several tasks, such as assessing robustness to changes in texture rotation, scale, and illumination, and distinguishing color textures, material textures, and texture attributes. The goal is to understand the potential and differences among these models when directly applied to texture recognition, using pre-trained ViTs primarily for feature extraction and employing linear classifiers for evaluation. We also evaluate their efficiency, which is one of the main drawbacks in contrast to other methods. Our results show that ViTs generally outperform both CNNs and hand-engineered models, especially when using stronger pre-training and tasks involving in-the-wild textures (images from the internet). We highlight the following promising models: ViT-B with DINO pre-training, BeiTv2, and the Swin architecture, as well as the EfficientFormer as a low-cost alternative. In terms of efficiency, although having a higher number of GFLOPs and parameters, ViT-B and BeiT(v2) can achieve a lower feature extraction time on GPUs compared to ResNet50.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 39 of its 70 references that are in the map. First posted on arXiv 10 Jun 2024.

Builds on

39 of 70 references.

References · 39

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  7. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  8. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  9. 2013Describing Textures in the Wildcited by 2,110
  10. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
  11. 2021Going deeper with Image Transformerscited by 759
  12. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 671
27 more
  1. 2021MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformercited by 636
  2. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 516
  3. 2021ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biasescited by 484
  4. 2013DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognitioncited by 422
  5. 2022DeiT III: Revenge of the ViTcited by 417
  6. 2020Learning Texture Transformer Network for Image Super-Resolutioncited by 346
  7. 2021XCiT: Cross-Covariance Image Transformerscited by 335
  8. 2022BEiT v2: Masked Image Modeling with Vector-Quantized Visual Tokenizerscited by 298
  9. 2021When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentationscited by 259
  10. 2021Knowledge distillation: A good teacher is patient and consistentcited by 255
  11. 2022EfficientFormer: Vision Transformers at MobileNet Speedcited by 240
  12. 2021Co-Scale Conv-Attentional Image Transformerscited by 224
  13. 2021Are Transformers More Robust Than CNNs?cited by 187
  14. 2021You Only Look at One Sequence: Rethinking Transformer in Vision through Object Detectioncited by 170
  15. 2022Separable Self-attention for Mobile Vision Transformerscited by 163
  16. 2021Transformer for Single Image Super-Resolutioncited by 150
  17. 2022Global Context Vision Transformerscited by 98
  18. 2021Mixed Transformer U-Net For Medical Image Segmentationcited by 95
  19. 2023Getting ViT in Shape: Scaling Laws for Compute-Optimal Model Designcited by 87
  20. 2022Next-ViT: Next Generation Vision Transformer for Efficient Deployment in Realistic Industrial Scenarioscited by 65
  21. 2024SHViT: Single-Head Vision Transformer with Memory Efficient Macro Designcited by 55
  22. 2016Deep TEN: Texture Encoding Networkcited by 44
  23. 2018From BoW to CNN: Two Decades of Texture Representation for Texture Classificationcited by 39
  24. 2023RADAM: Texture Recognition through Randomized Aggregated Encoding of Deep Activation Mapscited by 12
  25. 2019Spatio-spectral networks for color-texture analysiscited by 5
  26. 2018Multilayer Complex Network Descriptors for Color-Texture Characterizationcited by 3
  27. 2021Construction material classification on imbalanced datasets using Vision Transformer (ViT) architecturecited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 2.

6 papers.

  1. 2025USF-MAE: Ultrasound Self-Supervised Foundation Model with Masked Autoencodingcited by 16
  2. 2025VORTEX: Challenging CNNs at Texture Recognition by using Vision Transformers with Orderless and Randomized Token Encodingscited by 1
  3. 2025Towards Automated Semantic Interpretability in Reinforcement Learning via Vision-Language Modelscited by 1
  4. 2025Exploring Diagnostic Prompting Approach for Multimodal LLM-based Visual Complexity Assessment: A Case Study of Amazon Search Result Pagescited by 0
  5. 2026Chaotic Contrastive Learning for Robust Texture Classificationcited by 0
  6. 2026A self-supervised learning approach to deep filter banks for texture recognitioncited by 0