mlsanityData to 14 Oct 2026

TCFormer: Visual Recognition via Token Clustering Transformer

arXiv 2407.11321 ↗First posted 16 Jul 2024cs.CVcited by 12

Not on the map. Why?

Abstract (arXiv)

Transformers are widely used in computer vision areas and have achieved remarkable success. Most state-of-the-art approaches split images into regular grids and represent each grid region with a vision token. However, fixed token distribution disregards the semantic meaning of different image regions, resulting in sub-optimal performance. To address this issue, we propose the Token Clustering Transformer (TCFormer), which generates dynamic vision tokens based on semantic meaning. Our dynamic tokens possess two crucial characteristics: (1) Representing image regions with similar semantic meanings using the same vision token, even if those regions are not adjacent, and (2) concentrating on regions with valuable details and represent them using fine tokens. Through extensive experimentation across various applications, including image classification, human pose estimation, semantic segmentation, and object detection, we demonstrate the effectiveness of our TCFormer. The code and models for this work are available at https://github.com/zengwang430521/TCFormer.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 71 of its 82 references that are in the map. First posted on arXiv 16 Jul 2024.

Builds on

71 of 82 references.

References · 71

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  6. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  7. 2020End-to-End Object Detection with Transformerscited by 7,414
  8. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  9. 2017Mask R-CNNcited by 7,277
  10. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  11. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  12. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
38 more
  1. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  2. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  3. 2014Going Deeper with Convolutionscited by 4,241
  4. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  5. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  6. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
  7. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
  8. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  9. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,962
  10. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,385
  11. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  12. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
  13. 2017Random Erasing Data Augmentationcited by 1,239
  14. 2019Objects as Pointscited by 1,175
  15. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,162
  16. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,101
  17. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,064
  18. 2022Token Merging: Your ViT But Fastercited by 1,021
  19. 2020Designing Network Design Spacescited by 960
  20. 2021CvT: Introducing Convolutions to Vision Transformerscited by 947
  21. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 915
  22. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
  23. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  24. 2021Going deeper with Image Transformerscited by 764
  25. 2021PVT v2: Improved Baselines with Pyramid Vision Transformercited by 751
  26. 2019Panoptic Feature Pyramid Networkscited by 645
  27. 2021Transformer in Transformercited by 633
  28. 2021Twins: Revisiting the Design of Spatial Attention in Vision Transformerscited by 565
  29. 2022Vision Transformer Adapter for Dense Predictionscited by 565
  30. 2021MetaFormer Is Actually What You Need for Visioncited by 524
  31. 2018Simple Baselines for Human Pose Estimation and Trackingcited by 502
  32. 2022ViTPose: Simple Vision Transformer Baselines for Human Pose Estimationcited by 456
  33. 2017Cascaded Pyramid Network for Multi-Person Pose Estimationcited by 341
  34. 2013DeepPose: Human Pose Estimation via Deep Neural Networkscited by 305
  35. 2017Integral Human Pose Regressioncited by 256
  36. 2019HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimationcited by 247
  37. 2021LocalViT: Analyzing Locality in Vision Transformerscited by 228
  38. 2021AdaViT: Adaptive Vision Transformers for Efficient Image Recognitioncited by 228

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 6.

12 papers.

  1. 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inferencecited by 391
  2. 2026Multimodal Model for Computational Pathology:Representation Learning and Image Compressioncited by 6
  3. 2025H2OT: Hierarchical Hourglass Tokenizer for Efficient Video Pose Transformerscited by 5
  4. 2025Native Segmentation Vision Transformerscited by 2
  5. 2025Towards Ancient Plant Seed Classification: A Benchmark Dataset and Baseline Modelcited by 2
  6. 2025FTCFormer: Fuzzy Token Clustering Transformer for Image Classificationcited by 1
  7. 2026MaMe & MaRe: Matrix-Based Token Merging and Restoration for Efficient Visual Perception and Synthesiscited by 1
  8. 2026Revisiting Token Compression for Accelerating ViT-based Sparse Multi-View 3D Object Detectorscited by 1
  9. 2026Hyperspectral Image Classification via Efficient Global Spectral Supertoken Clusteringcited by 1
  10. 2025Where Do Tokens Go? Understanding Pruning Behaviors in STEP at High Resolutionscited by 0
  11. 2026Do Vision Transformers Need All-to-All Attention? Global Communication Through Elastic Learned Corescited by 0
  12. 2026Frequency Decoupled Framework for Screen Content Image Super-Resolutioncited by 0