mlsanityData to 7 Oct 2026

Vision Transformers for Mobile Applications: A Short Survey

arXiv 2305.19365 ↗First posted 30 May 2023cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Vision Transformers (ViTs) have demonstrated state-of-the-art performance on many Computer Vision Tasks. Unfortunately, deploying these large-scale ViTs is resource-consuming and impossible for many mobile devices. While most in the community are building for larger and larger ViTs, we ask a completely opposite question: How small can a ViT be within the tradeoffs of accuracy and inference latency that make it suitable for mobile deployment? We look into a few ViTs specifically designed for mobile applications and observe that they modify the transformer's architecture or are built around the combination of CNN and transformer. Recent work has also attempted to create sparse ViT networks and proposed alternatives to the attention module. In this paper, we study these architectures, identify the challenges and analyze what really makes a vision transformer suitable for mobile applications. We aim to serve as a baseline for future research direction and hopefully lay the foundation to choose the exemplary vision transformer architecture for your application running on mobile devices.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 36 of its 47 references that are in the map. First posted on arXiv 30 May 2023.

Builds on

36 of 47 references.

References · 36

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  3. 2020End-to-End Object Detection with Transformerscited by 7,384
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  5. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  6. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  7. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  8. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  9. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  10. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  11. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  12. 2021Video Swin Transformercited by 1,090
24 more
  1. 2022Token Merging: Your ViT But Fastercited by 995
  2. 2020A Survey on Visual Transformercited by 653
  3. 2021MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformercited by 636
  4. 2021MetaFormer Is Actually What You Need for Visioncited by 523
  5. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 516
  6. 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
  7. 2019The State of Sparsity in Deep Neural Networkscited by 471
  8. 2023Hyena Hierarchy: Towards Larger Convolutional Language Modelscited by 388
  9. 2021ResMLP: Feedforward networks for image classification with data-efficient trainingcited by 338
  10. 2022EfficientFormer: Vision Transformers at MobileNet Speedcited by 240
  11. 2021Mobile-Former: Bridging MobileNet and Transformercited by 234
  12. 2021AdaViT: Adaptive Vision Transformers for Efficient Image Recognitioncited by 228
  13. 2021Chasing Sparsity in Vision Transformers: An End-to-End Explorationcited by 181
  14. 2021Adaptive Token Sampling For Efficient Vision Transformerscited by 181
  15. 2022A Review of Sparse Expert Models in Deep Learningcited by 147
  16. 2021Group Fisher Pruning for Practical Network Compressioncited by 130
  17. 2022EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applicationscited by 121
  18. 2021An Attention Free Transformercited by 111
  19. 2022I-ViT: Integer-only Quantization for Efficient Vision Transformer Inferencecited by 93
  20. 2022Next-ViT: Next Generation Vision Transformer for Efficient Deployment in Realistic Industrial Scenarioscited by 65
  21. 2022MixFormer: Mixing Features across Windows and Dimensionscited by 43
  22. 2017Compressing DMA Engine: Leveraging Activation Sparsity for Training Deep Neural Networkscited by 36
  23. 2023Ten Lessons We Have Learned in the New "Sparseland": A Short Handbook for Sparse Neural Network Researcherscited by 25
  24. 2019Taxonomy and Evaluation of Structured Compression of Convolutional Neural Networkscited by 12

Built on by

Not cited yet.