mlsanityData to 7 Oct 2026

EfficientFormer: Vision Transformers at MobileNet Speed

arXiv 2206.01191 ↗First posted 2 Jun 2022cs.CVcited by 240

Not on the map. Why?

Abstract (arXiv)

Vision Transformers (ViT) have shown rapid progress in computer vision tasks, achieving promising results on various benchmarks. However, due to the massive number of parameters and model design, textite.g., attention mechanism, ViT-based models are generally times slower than lightweight convolutional networks. Therefore, the deployment of ViT for real-time applications is particularly challenging, especially on resource-constrained hardware such as mobile devices. Recent efforts try to reduce the computation complexity of ViT through network architecture search or hybrid design with MobileNet block, yet the inference speed is still unsatisfactory. This leads to an important question: can transformers run as fast as MobileNet while obtaining high performance? To answer this, we first revisit the network architecture and operators used in ViT-based models and identify inefficient designs. Then we introduce a dimension-consistent pure transformer (without MobileNet blocks) as a design paradigm. Finally, we perform latency-driven slimming to get a series of final models dubbed EfficientFormer. Extensive experiments show the superiority of EfficientFormer in performance and speed on mobile devices. Our fastest model, EfficientFormer-L1, achieves 79.2% top-1 accuracy on ImageNet-1K with only 1.6 ms inference latency on iPhone 12 (compiled with CoreML), which runs as fast as MobileNetV2× 1.4 (1.6 ms, 74.7% top-1), and our largest model, EfficientFormer-L7, obtains 83.3% accuracy with only 7.0 ms latency. Our work proves that properly designed transformers can reach extremely low latency on mobile devices while maintaining high performance.

Key facts. Cited by 240 papers in the map (data through 7 Oct 2026). Builds on 72 of its 87 references that are in the map. First posted on arXiv 2 Jun 2022.

Builds on

72 of 87 references.

References · 72

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  7. 2020End-to-End Object Detection with Transformerscited by 7,384
  8. 2017Mask R-CNNcited by 7,259
  9. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  10. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  11. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  12. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
38 more
  1. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  2. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  3. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  4. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  5. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  6. 2019Searching for MobileNetV3cited by 1,984
  7. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  8. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  9. 2020Reformer: The Efficient Transformercited by 1,658
  10. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  11. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,215
  12. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
  13. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  14. 2021Video Swin Transformercited by 1,090
  15. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
  16. 2020Designing Network Design Spacescited by 954
  17. 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
  18. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
  19. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 849
  20. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
  21. 2021Going deeper with Image Transformerscited by 759
  22. 2019Panoptic Feature Pyramid Networkscited by 644
  23. 2021MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformercited by 636
  24. 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 566
  25. 2021MetaFormer Is Actually What You Need for Visioncited by 523
  26. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 516
  27. 2022MaxViT: Multi-Axis Vision Transformercited by 476
  28. 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
  29. 2022DeiT III: Revenge of the ViTcited by 417
  30. 2021Rethinking Spatial Dimensions of Vision Transformerscited by 361
  31. 2021ResMLP: Feedforward networks for image classification with data-efficient trainingcited by 338
  32. 2021CMT: Convolutional Neural Networks Meet Vision Transformerscited by 310
  33. 2021DeepViT: Towards Deeper Vision Transformercited by 288
  34. 2021Escaping the Big Data Paradigm with Compact Transformerscited by 261
  35. 2021Incorporating Convolution Designs into Visual Transformerscited by 248
  36. 2021Mobile-Former: Bridging MobileNet and Transformercited by 234
  37. 2021LocalViT: Analyzing Locality in Vision Transformerscited by 228
  38. 2021AdaViT: Adaptive Vision Transformers for Efficient Image Recognitioncited by 228

50 most cited of 72 All 72 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 17; 2023: 60; 2024: 80; 2025: 50; 2026: 33.

240 papers.

Sort:

By area:

  1. 2023Faster Segment Anything: Towards Lightweight SAM for Mobile Applicationscited by 345
  2. 2023SnapFusion: Text-to-Image Diffusion Model on Mobile Devices within Two Secondscited by 227
  3. 2024MambaVision: A Hybrid Mamba-Transformer Vision Backbonecited by 222
  4. 2023EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anythingcited by 187
  5. 2023EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attentioncited by 172
  6. 2022Separable Self-attention for Mobile Vision Transformerscited by 163
  7. 2023FastViT: A Fast Hybrid Vision Transformer using Structural Reparameterizationcited by 161
  8. 2024EfficientVMamba: Atrous Selective Scan for Light Weight Visual Mambacited by 146
  9. 2023Spike-driven Transformercited by 141
  10. 2023Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networkscited by 129
  11. 2023RepViT: Revisiting Mobile CNN From ViT Perspectivecited by 123
  12. 2022Rethinking Vision Transformers for MobileNet Size and Speedcited by 121
38 more
  1. 2024MobileNetV4 -- Universal Models for the Mobile Ecosystemcited by 98
  2. 2023MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Trainingcited by 96
  3. 2024Efficient Multimodal Large Language Models: A Surveycited by 95
  4. 2024A Survey of Resource-efficient LLM and Multimodal Foundation Modelscited by 84
  5. 2024SiMBA: Simplified Mamba-Based Architecture for Vision and Multivariate Time seriescited by 81
  6. 2023Full Stack Optimization of Transformer Inference: a Surveycited by 79
  7. 2023DiffRate : Differentiable Compression Rate for Efficient Vision Transformerscited by 78
  8. 2023SwiftFormer: Efficient Additive Attention for Transformer-based Real-time Mobile Vision Applicationscited by 66
  9. 2022Next-ViT: Next Generation Vision Transformer for Efficient Deployment in Realistic Industrial Scenarioscited by 65
  10. 2023FasterViT: Fast Vision Transformers with Hierarchical Attentioncited by 65
  11. 2024Mamba-360: Survey of State Space Models as Transformer Alternative for Long Sequence Modelling: Methods, Applications, and Challengescited by 63
  12. 2022MogaNet: Multi-order Gated Aggregation Networkcited by 59
  13. 2024SHViT: Single-Head Vision Transformer with Memory Efficient Macro Designcited by 55
  14. 2023See and Think: Embodied Agent in Virtual Environmentcited by 54
  15. 2023Scale-Aware Modulation Meet Transformercited by 49
  16. 2022Real-Time Neural Light Field on Mobile Devicescited by 48
  17. 2024Good Seed Makes a Good Crop: Discovering Secret Seeds in Text-to-Image Diffusion Modelscited by 47
  18. 2023Adaptive Frequency Filters As Efficient Global Token Mixerscited by 45
  19. 2024MambaMixer: Efficient Selective State Space Models with Dual Token and Channel Selectioncited by 40
  20. 2024ViTamin: Designing Scalable Vision Models in the Vision-Language Eracited by 39
  21. 2023Rethinking Mobile Block for Efficient Attention-based Modelscited by 36
  22. 2023MobileViG: Graph-Based Sparse Attention for Mobile Vision Applicationscited by 35
  23. 2023A Survey of Techniques for Optimizing Transformer Inferencecited by 35
  24. 2023Black-box Membership Inference Attacks against Fine-tuned Diffusion Modelscited by 35
  25. 2023Dynamic Perceiver for Efficient Visual Recognitioncited by 33
  26. 2023FP8 versus INT8 for efficient deep learning inferencecited by 31
  27. 2022BiViT: Extremely Compressed Binary Vision Transformercited by 28
  28. 2022Castling-ViT: Compressing Self-Attention via Switching Towards Linear-Angular Attention at Vision Transformer Inferencecited by 28
  29. 2023Lightweight Vision Transformer with Bidirectional Interactioncited by 25
  30. 2022Fast-iTPN: Integrally Pre-Trained Transformer Pyramid Network with Token Migrationcited by 24
  31. 2023TinyMIM: An Empirical Study of Distilling MIM Pre-trained Modelscited by 24
  32. 2023Localizing Moments in Long Video Via Multimodal Guidancecited by 23
  33. 2025Similarity-Guided Layer-Adaptive Vision Transformer for UAV Trackingcited by 22
  34. 2023TransXNet: Learning Both Global and Local Dynamics with a Dual Dynamic Token Mixer for Visual Recognitioncited by 21
  35. 2024GreedyViG: Dynamic Axial Graph Construction for Efficient Vision GNNscited by 21
  36. 2024Isomorphic Pruning for Vision Modelscited by 21
  37. 2022CAP: Correlation-Aware Pruning for Highly-Accurate Sparse Vision Modelscited by 20
  38. 2024ELGC-Net: Efficient Local-Global Context Aggregation for Remote Sensing Change Detectioncited by 19

50 most cited of 240 All 240 (JSON)