mlsanityData to 7 Oct 2026

CF-ViT: A General Coarse-to-Fine Method for Vision Transformer

arXiv 2203.03821 ↗First posted 8 Mar 2022cs.CVcited by 63

Not on the map. Why?

Abstract (arXiv)

Vision Transformers (ViT) have made many breakthroughs in computer vision tasks. However, considerable redundancy arises in the spatial dimension of an input image, leading to massive computational costs. Therefore, We propose a coarse-to-fine vision transformer (CF-ViT) to relieve computational burden while retaining performance in this paper. Our proposed CF-ViT is motivated by two important observations in modern ViT models: (1) The coarse-grained patch splitting can locate informative regions of an input image. (2) Most images can be well recognized by a ViT model in a small-length token sequence. Therefore, our CF-ViT implements network inference in a two-stage manner. At coarse inference stage, an input image is split into a small-length patch sequence for a computationally economical classification. If not well recognized, the informative patches are identified and further re-split in a fine-grained granularity. Extensive experiments demonstrate the efficacy of our CF-ViT. For example, without any compromise on performance, CF-ViT reduces 53% FLOPs of LV-ViT, and also achieves 2.01x throughput.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). Builds on 37 of its 50 references that are in the map. First posted on arXiv 8 Mar 2022.

Builds on

37 of 50 references.

References · 37

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  5. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  6. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  7. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  8. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  9. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  10. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 916
  11. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 849
  12. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
25 more
  1. 2021Going deeper with Image Transformerscited by 759
  2. 2020A Survey on Visual Transformercited by 653
  3. 2021Transformer in Transformercited by 632
  4. 2021Twins: Revisiting the Design of Spatial Attention in Vision Transformerscited by 565
  5. 2021Dynamic Neural Networks: A Surveycited by 405
  6. 2021Conditional Positional Encodings for Vision Transformerscited by 402
  7. 2021Rethinking Spatial Dimensions of Vision Transformerscited by 361
  8. 2021Incorporating Convolution Designs into Visual Transformerscited by 248
  9. 2021LocalViT: Analyzing Locality in Vision Transformerscited by 228
  10. 2021Co-Scale Conv-Attentional Image Transformerscited by 224
  11. 2021Evo-ViT: Slow-Fast Token Evolution for Dynamic Vision Transformercited by 175
  12. 2020Resolution Adaptive Networks for Efficient Inferencecited by 146
  13. 2021All Tokens Matter: Token Labeling for Training Better Vision Transformerscited by 144
  14. 2021Not All Images are Worth 16x16 Words: Dynamic Transformers for Efficient Image Recognitioncited by 143
  15. 2021Patch Slimming for Efficient Vision Transformerscited by 131
  16. 2022QuadTree Attention for Vision Transformerscited by 129
  17. 2021IA-RED2: Interpretability-Aware Redundancy Reduction for Vision Transformerscited by 127
  18. 2021Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformercited by 115
  19. 2020Glance and Focus: a Dynamic Approach to Reducing Spatial Redundancy in Image Classificationcited by 99
  20. 2021Scalable Vision Transformers with Hierarchical Poolingcited by 84
  21. 2021Adaptive Focus for Efficient Video Recognitioncited by 76
  22. 2021MSG-Transformer: Exchanging Local Spatial Information by Manipulating Messenger Tokenscited by 46
  23. 2021Glance-and-Gaze Vision Transformercited by 42
  24. 2021AdaFocus V2: End-to-End Training of Spatial Dynamic Networks for Video Recognitioncited by 37
  25. 2021Self-slimmed Vision Transformercited by 27

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 10; 2024: 15; 2025: 20; 2026: 14.

63 papers.

Sort:

By area:

  1. 2024Boosting Multimodal Large Language Models with Visual Tokens Withdrawal for Rapid Inferencecited by 137
  2. 2023DiffRate : Differentiable Compression Rate for Efficient Vision Transformerscited by 78
  3. 2022MogaNet: Multi-order Gated Aggregation Networkcited by 59
  4. 2023Which Tokens to Use? Investigating Token Reduction in Vision Transformerscited by 59
  5. 2022ResFormer: Scaling ViTs with Multi-Resolution Trainingcited by 34
  6. 2024ALGM: Adaptive Local-then-Global Token Merging for Efficient Semantic Segmentation with Plain Vision Transformerscited by 28
  7. 2024ScanFormer: Referring Expression Comprehension by Iteratively Scanningcited by 18
  8. 2023Jointly-Learned Exit and Inference for a Dynamic Neural Network : JEI-DNNcited by 15
  9. 2022Super Vision Transformercited by 13
  10. 2023Bi-ViT: Pushing the Limit of Vision Transformer Quantizationcited by 13
  11. 2023Leveraging Vision-Centric Multi-Modal Expertise for 3D Object Detectioncited by 13
  12. 2024LF-ViT: Reducing Spatial Redundancy in Vision Transformer for Efficient Image Recognitioncited by 13
38 more
  1. 2024Agglomerative Token Clusteringcited by 13
  2. 2025Constructive Distortion: Improving MLLMs with Attention-Guided Image Warpingcited by 13
  3. 2025Accelerating Vision Transformers with Adaptive Patch Sizescited by 12
  4. 2023SHaRPose: Sparse High-Resolution Representation for Human Pose Estimationcited by 11
  5. 2022SMMix: Self-Motivated Image Mixing for Vision Transformerscited by 8
  6. 2024AFter: Attention-based Fusion Router for RGBT Trackingcited by 7
  7. 2023Content-Adaptive Downsampling in Convolutional Neural Networkscited by 5
  8. 2023TransFusionOdom: Interpretable Transformer-based LiDAR-Inertial Fusion Odometry Estimationcited by 4
  9. 2024Tiny Models are the Computational Saver for Large Modelscited by 4
  10. 2024Frequency-Guided Masking for Enhanced Vision Self-Supervised Learningcited by 4
  11. 2025Sparse Model Inversion: Efficient Inversion of Vision Transformers for Data-Free Applicationscited by 4
  12. 2024FViT: A Focal Vision Transformer with Gabor Filtercited by 3
  13. 2024DyCE: Dynamically Configurable Exiting for Deep Learning Compression and Real-time Scalingcited by 3
  14. 2024Semantic Alignment and Reinforcement for Data-Free Quantization of Vision Transformerscited by 3
  15. 2026DDiT: Dynamic Patch Scheduling for Efficient Diffusion Transformerscited by 3
  16. 2024Homogeneous Tokenizer Matters: Homogeneous Visual Tokenizer for Remote Sensing Image Understandingcited by 2
  17. 2024DocParseNet: Advanced Semantic Segmentation and OCR Embeddings for Efficient Scanned Document Annotationcited by 2
  18. 2025Image Recognition with Online Lightweight Vision Transformer: A Surveycited by 2
  19. 2025Dynamic Granularity Matters: Rethinking Vision Transformers Beyond Fixed Patch Splittingcited by 2
  20. 2026Fusion: A Framework for Unified Sequential Token AdaptatIon in VisiOn TraNsformerscited by 2
  21. 2023EViT: An Eagle Vision Transformer with Bi-Fovea Self-Attentioncited by 1
  22. 2024LoDisc: Learning Global-Local Discriminative Features for Self-Supervised Fine-Grained Visual Recognitioncited by 1
  23. 2025SAC-ViT: Semantic-Aware Clustering Vision Transformer with Early Exitcited by 1
  24. 2025CF-DETR: Coarse-to-Fine Transformer for Real-Time Object Detectioncited by 1
  25. 2025Consistent Assistant Domains Transformer for Source-free Domain Adaptationcited by 1
  26. 2025Image Complexity-Aware Adaptive Retrieval for Efficient Vision-Language Modelscited by 1
  27. 2026Adaptive Volumetric Mechanical Property Fields Invariant to Resolutioncited by 1
  28. 2023Recent Advances of Local Mechanisms in Computer Vision: A Survey and Outlook of Recent Workcited by 0
  29. 2024ImagePiece: Content-aware Re-tokenization for Efficient Image Recognitioncited by 0
  30. 2025ORXE: Orchestrating Experts for Dynamically Configurable Efficiencycited by 0
  31. 2025Contextually Guided Transformers via Low-Rank Adaptationcited by 0
  32. 2025PoseGRAF: Geometric-Reinforced Adaptive Fusion for Monocular 3D Human Pose Estimationcited by 0
  33. 2025Boosting Adversarial Transferability Against Defenses via Multi-Scale Transformationcited by 0
  34. 2025TinyDrop: Tiny Model Guided Token Dropping for Vision Transformerscited by 0
  35. 2025Where Do Tokens Go? Understanding Pruning Behaviors in STEP at High Resolutionscited by 0
  36. 2025Parallel Vision Token Scheduling for Fast and Accurate Multimodal LMMs Inferencecited by 0
  37. 2025Granular Computing-driven SAM: From Coarse-to-Fine Guidance for Prompt-Free Segmentationcited by 0
  38. 2025Towards an Effective Action-Region Tracking Framework for Fine-grained Video Action Recognitioncited by 0

50 most cited of 63 All 63 (JSON)