mlsanityData to 14 Oct 2026

CascadedViT: Cascaded Chunk-FeedForward and Cascaded Group Attention Vision Transformer

arXiv 2511.14111 ↗First posted 18 Nov 2025cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Vision Transformers (ViTs) have demonstrated remarkable performance across a range of computer vision tasks; however, their high computational, memory, and energy demands hinder deployment on resource-constrained platforms. In this paper, we propose Cascaded-ViT (CViT), a lightweight and compute-efficient vision transformer architecture featuring a novel feedforward network design called Cascaded-Chunk Feed Forward Network (CCFFN). By splitting input features, CCFFN improves parameter and FLOP efficiency without sacrificing accuracy. Experiments on ImageNet-1K show that our CViT-XL model achieves 75.5% Top-1 accuracy while reducing FLOPs by 15% and energy consumption by 3.3% compared to EfficientViT-M5. Across various model sizes, the CViT family consistently exhibits the lowest energy consumption, making it suitable for deployment on battery-constrained devices such as mobile phones and drones. Furthermore, when evaluated using a new metric called Accuracy-Per-FLOP (APF), which quantifies compute efficiency relative to accuracy, CViT models consistently achieve top-ranking efficiency. Particularly, CViT-L is 2.2% more accurate than EfficientViT-M2 while having comparable APF scores.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 55 of its 74 references that are in the map. First posted on arXiv 18 Nov 2025.

Builds on

55 of 74 references.

References · 55

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2017Mask R-CNNcited by 7,277
  6. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  7. 2020Scaling Laws for Neural Language Modelscited by 6,586
  8. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  9. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  10. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  11. 2017Squeeze-and-Excitation Networkscited by 4,789
  12. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
38 more
  1. 2019Searching for MobileNetV3cited by 1,994
  2. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,962
  3. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  4. 2017Random Erasing Data Augmentationcited by 1,239
  5. 2021CvT: Introducing Convolutions to Vision Transformerscited by 947
  6. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
  7. 2019Green AIcited by 627
  8. 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
  9. 2017AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baselinecited by 373
  10. 2022EfficientFormer: Vision Transformers at MobileNet Speedcited by 241
  11. 2022TinyViT: Fast Pretraining Distillation for Small Vision Transformerscited by 234
  12. 2022Knowledge Distillation from A Stronger Teachercited by 208
  13. 2023EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attentioncited by 172
  14. 2021SPViT: Enabling Faster Vision Transformers via Soft Token Pruningcited by 167
  15. 2022Separable Self-attention for Mobile Vision Transformerscited by 163
  16. 2021Patch Slimming for Efficient Vision Transformerscited by 133
  17. 2022EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applicationscited by 121
  18. 2022EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformerscited by 116
  19. 2021Scale Efficiently: Insights from Pre-training and Fine-tuning Transformerscited by 106
  20. 2017Soft Weight-Sharing for Neural Network Compressioncited by 99
  21. 2022MiniViT: Compressing Vision Transformers with Weight Multiplexingcited by 88
  22. 2021Vision Transformer Pruningcited by 79
  23. 2021Global Vision Transformer Pruning with Hessian-Aware Saliencycited by 70
  24. 2024SHViT: Single-Head Vision Transformer with Memory Efficient Macro Designcited by 56
  25. 2024A Survey on Transformer Compressioncited by 46
  26. 2022MobileViTv3: Mobile-Friendly Vision Transformer with Simple and Effective Fusion of Local, Global and Input Featurescited by 44
  27. 2022LightViT: Towards Light-Weight Convolution-Free Vision Transformerscited by 40
  28. 2023A survey of the Vision Transformers and their CNN-Transformer based Variantscited by 33
  29. 2022Reversible Column Networkscited by 32
  30. 2020Deeper Insights into Weight Sharing in Neural Architecture Searchcited by 24
  31. 2025Learning Occlusion-Robust Vision Transformers for Real-Time UAV Trackingcited by 22
  32. 2023One Wide Feedforward is All You Needcited by 15
  33. 2024Vision Transformers for End-to-End Vision-Based Quadrotor Obstacle Avoidancecited by 14
  34. 2021Dancing along Battery: Enabling Transformer with Run-time Reconfigurability on Mobile Devicescited by 9
  35. 2025Analytical FFN-to-MoE Restructuring via Activation Pattern Analysiscited by 8
  36. 2025MicroViT: A Vision Transformer with Low Complexity Self Attention for Edge Devicecited by 8
  37. 2024Double-Exponential Increases in Inference Energy: The Cost of the Race for Accuracycited by 6
  38. 2023Can pruning make Large Language Models more efficient?cited by 5

50 most cited of 55 All 55 (JSON)

Built on by

Not cited yet.