mlsanityData to 14 Oct 2026

FasterViT: Fast Vision Transformers with Hierarchical Attention

arXiv 2306.06189 ↗First posted 9 Jun 2023cs.CV cs.AI cs.LGcited by 65

Not on the map. Why?

Abstract (arXiv)

We design a new family of hybrid CNN-ViT neural networks, named FasterViT, with a focus on high image throughput for computer vision (CV) applications. FasterViT combines the benefits of fast local representation learning in CNNs and global modeling properties in ViT. Our newly introduced Hierarchical Attention (HAT) approach decomposes global self-attention with quadratic complexity into a multi-level attention with reduced computational costs. We benefit from efficient window-based self-attention. Each window has access to dedicated carrier tokens that participate in local and global representation learning. At a high level, global self-attentions enable the efficient cross-window communication at lower costs. FasterViT achieves a SOTA Pareto-front in terms of accuracy and image throughput. We have extensively validated its effectiveness on various CV tasks including classification, object detection and segmentation. We also show that HAT can be used as a plug-and-play module for existing networks and enhance them. We further demonstrate significantly faster and more accurate performance than competitive counterparts for images with high resolution. Code is available at https://github.com/NVlabs/FasterViT.

Key facts. Cited by 65 papers in the map (data through 14 Oct 2026). Builds on 68 of its 83 references that are in the map. First posted on arXiv 9 Jun 2023.

Builds on

68 of 83 references.

References · 68

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  7. 2017Mask R-CNNcited by 7,277
  8. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  9. 2016Layer Normalizationcited by 5,669
  10. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  11. 2022A ConvNet for the 2020scited by 3,579
  12. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
38 more
  1. 2020Longformer: The Long-Document Transformercited by 3,413
  2. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  3. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  4. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
  5. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,962
  6. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  7. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,850
  8. 2020Big Bird: Transformers for Longer Sequencescited by 1,684
  9. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,515
  10. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
  11. 2019Natural Adversarial Examplescited by 1,408
  12. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
  13. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,271
  14. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,228
  15. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,153
  16. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  17. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,135
  18. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,101
  19. 2020Designing Network Design Spacescited by 960
  20. 2021Multiscale Vision Transformerscited by 897
  21. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
  22. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
  23. 2021Going deeper with Image Transformerscited by 764
  24. 2021PVT v2: Improved Baselines with Pyramid Vision Transformercited by 751
  25. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 675
  26. 2019Large Batch Optimization for Deep Learning: Training BERT in 76 minutescited by 674
  27. 2021Transformer in Transformercited by 633
  28. 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 569
  29. 2021Twins: Revisiting the Design of Spatial Attention in Vision Transformerscited by 565
  30. 2021CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windowscited by 527
  31. 2021MetaFormer Is Actually What You Need for Visioncited by 524
  32. 2019Hybrid Task Cascade for Instance Segmentationcited by 481
  33. 2022MaxViT: Multi-Axis Vision Transformercited by 476
  34. 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
  35. 2022DeiT III: Revenge of the ViTcited by 419
  36. 2021Conditional Positional Encodings for Vision Transformerscited by 406
  37. 2021ResNet strikes back: An improved training procedure in timmcited by 368
  38. 2021AdaViT: Adaptive Tokens for Efficient Vision Transformercited by 362

50 most cited of 68 All 68 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 22; 2025: 22; 2026: 16.

65 papers.

Sort:

By area:

  1. 2024MambaVision: A Hybrid Mamba-Transformer Vision Backbonecited by 224
  2. 2024Spike-driven Transformer V2: Meta Spiking Neural Network Architecture Inspiring the Design of Next-generation Neuromorphic Chipscited by 103
  3. 2023MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Trainingcited by 96
  4. 2024Rewrite the Starscited by 55
  5. 2024XoFTR: Cross-modal Feature Matching Transformercited by 48
  6. 2024A Survey on Transformer Compressioncited by 46
  7. 2024LW-DETR: A Transformer Replacement to YOLO for Real-Time Detectioncited by 36
  8. 2025ViT3: Unlocking Test-Time Training in Visioncited by 18
  9. 2024Embedding-Free Transformer with Inference Spatial Reduction for Efficient Semantic Segmentationcited by 14
  10. 2024A Lightweight Sparse Focus Transformer for Remote Sensing Image Change Captioningcited by 13
  11. 2023Low-Resolution Self-Attention for Semantic Segmentationcited by 12
  12. 2024CAS-ViT: Convolutional Additive Self-attention Vision Transformers for Efficient Mobile Applicationscited by 12
38 more
  1. 2025FlexiDiT: Your Diffusion Transformer Can Easily Generate High-Quality Samples with Less Computecited by 12
  2. 2023SBCFormer: Lightweight Network Capable of Full-size ImageNet Classification at 1 FPS on Single Board Computerscited by 9
  3. 2025iFormer: Integrating ConvNet and Transformer for Mobile Applicationcited by 9
  4. 2025HOTFormerLoc: Hierarchical Octree Transformer for Versatile Lidar Place Recognition Across Ground and Aerial Viewscited by 8
  5. 2024On Inductive Biases That Enable Generalization of Diffusion Transformerscited by 7
  6. 2024Vision Transformer with Sparse Scan Priorcited by 6
  7. 2024PADRe: A Unifying Polynomial Attention Drop-in Replacement for Efficient Vision Transformercited by 5
  8. 2024AsCAN: Asymmetric Convolution-Attention Networks for Efficient Recognition and Generationcited by 5
  9. 2025Enhancing Image Restoration Transformer via Adaptive Translation Equivariancecited by 5
  10. 2024ToSA: Token Selective Attention for Efficient Vision Transformerscited by 4
  11. 2024RepNeXt: A Fast Multi-Scale CNN using Structural Reparameterizationcited by 4
  12. 2025Atlas: Multi-Scale Attention Improves Long Context Image Modelingcited by 4
  13. 2025Vision Transformers are Circulant Attention Learnerscited by 4
  14. 2026SF-Mamba: Rethinking State Space Model for Visioncited by 4
  15. 2026EMFormer: Efficient Multi-Scale Transformer for Accumulative Context Weather Forecastingcited by 3
  16. 2026HAMSA: Scanning-Free Vision State Space Models via SpectralPulseNetcited by 3
  17. 2023ViR: Towards Efficient Vision Retention Backbonescited by 2
  18. 2025Rethinking Boundary Detection in Deep Learning-Based Medical Image Segmentationcited by 2
  19. 2025FLASH-D: FlashAttention with Hidden Softmax Divisioncited by 2
  20. 2025RoadFormer : Local-Global Feature Fusion for Road Surface Classification in Autonomous Drivingcited by 2
  21. 2025FastViDAR: Real-Time Omnidirectional Depth Estimation via Alternative Hierarchical Attentioncited by 2
  22. 2026VFA: Relieving Vector Operations in Flash Attention with Global Maximum Pre-computationcited by 2
  23. 2023TurboViT: Generating Fast Vision Transformers via Generative Architecture Searchcited by 1
  24. 2024HybridHash: Hybrid Convolutional and Self-Attention Deep Hashing for Image Retrievalcited by 1
  25. 2024FFNet: MetaMixer-based Efficient Convolutional Mixer Designcited by 1
  26. 2024Learning Spatial-Preserving Hierarchical Representations for Digital Pathologycited by 1
  27. 2024CTRL-F: Pairing Convolution with Transformer for Image Classification via Multi-Level Feature Cross-Attention and Representation Learning Fusioncited by 1
  28. 2024LMLT: Low-to-high Multi-Level Vision Transformer for Image Super-Resolutioncited by 1
  29. 2025SAC-ViT: Semantic-Aware Clustering Vision Transformer with Early Exitcited by 1
  30. 2025Learning Spatial Decay for Vision Transformerscited by 1
  31. 2025A Lightweight Convolution and Vision Transformer integrated model with Multi-scale Self-attention Mechanismcited by 1
  32. 2025MOBIUS: Big-to-Mobile Universal Instance Segmentation via Multi-modal Bottleneck Fusion and Calibrated Decoder Pruningcited by 1
  33. 2026Lost in the Hype: Revealing and Dissecting the Performance Degradation of Medical Multimodal Large Language Models in Image Classificationcited by 1
  34. 2024FAST GDRNPP: Improving the Speed of State-of-the-Art 6D Object Pose Estimationcited by 0
  35. 2024Chimera: A Block-Based Neural Architecture Search Framework for Event-Based Object Detectioncited by 0
  36. 2025A Multimodal Fusion Model Leveraging MLP Mixer and Handcrafted Features-based Deep Learning Networks for Facial Palsy Detectioncited by 0
  37. 2025Efficient Mixture of Geographical Species for On Device Wildlife Monitoringcited by 0
  38. 2025FORTRESS: Function-composition Optimized Real-Time Resilient Structural Segmentation via Kolmogorov-Arnold Enhanced Spatial Attention Networkscited by 0

50 most cited of 65 All 65 (JSON)