mlsanityData to 7 Oct 2026

Not All Tokens Are Equal: Human-centric Visual Analysis via Token Clustering Transformer

arXiv 2204.08680 ↗First posted 19 Apr 2022cs.CVcited by 99

Not on the map. Why?

Abstract (arXiv)

Vision transformers have achieved great successes in many computer vision tasks. Most methods generate vision tokens by splitting an image into a regular and fixed grid and treating each cell as a token. However, not all regions are equally important in human-centric vision tasks, e.g., the human body needs a fine representation with many tokens, while the image background can be modeled by a few tokens. To address this problem, we propose a novel Vision Transformer, called Token Clustering Transformer (TCFormer), which merges tokens by progressive clustering, where the tokens can be merged from different locations with flexible shapes and sizes. The tokens in TCFormer can not only focus on important areas but also adjust the token shapes to fit the semantic concept and adopt a fine resolution for regions containing critical details, which is beneficial to capturing detailed information. Extensive experiments show that TCFormer consistently outperforms its counterparts on different challenging human-centric tasks and datasets, including whole-body pose estimation on COCO-WholeBody and 3D human mesh reconstruction on 3DPW. Code is available at https://github.com/zengwang430521/TCFormer.git

Key facts. Cited by 99 papers in the map (data through 7 Oct 2026). Builds on 60 of its 83 references that are in the map. First posted on arXiv 19 Apr 2022.

Builds on

60 of 83 references.

References · 60

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  7. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  8. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  9. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  10. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  11. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  12. 2014Going Deeper with Convolutionscited by 4,231
38 more
  1. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  2. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  3. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,417
  4. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  5. 2017Random Erasing Data Augmentationcited by 1,235
  6. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,161
  7. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  8. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,061
  9. 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
  10. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 913
  11. 2017End-to-end Recovery of Human Shape and Posecited by 883
  12. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 849
  13. 2018Hierarchical Graph Representation Learning with Differentiable Poolingcited by 786
  14. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 625
  15. 2018Simple Baselines for Human Pose Estimation and Trackingcited by 502
  16. 2016Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervisioncited by 493
  17. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 417
  18. 2013DeepPose: Human Pose Estimation via Deep Neural Networkscited by 305
  19. 2020I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Imagecited by 274
  20. 20213D Human Pose Estimation with Spatial and Temporal Transformerscited by 271
  21. 2019Convolutional Mesh Regression for Single-Image Human Shape Reconstructioncited by 270
  22. 2021Mesh Graphormercited by 264
  23. 2020Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Posecited by 248
  24. 2019HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimationcited by 247
  25. 2016Associative Embedding: End-to-End Learning for Joint Detection and Groupingcited by 211
  26. 2020Whole-Body Human Pose Estimation in the Wildcited by 178
  27. 2021HRFormer: High-Resolution Transformer for Dense Predictioncited by 149
  28. 2018BodyNet: Volumetric Inference of 3D Human Body Shapescited by 141
  29. 2018Look at Boundary: A Boundary-Aware Face Alignment Algorithmcited by 137
  30. 2021Human Pose Regression with Residual Log-likelihood Estimationcited by 131
  31. 2020Exemplar Fine-Tuning for 3D Human Model Fitting Towards In-the-Wild 3D Human Pose Estimationcited by 113
  32. 2017Wing Loss for Robust Facial Landmark Localisation with Convolutional Neural Networkscited by 108
  33. 2021TokenPose: Learning Keypoint Tokens for Human Pose Estimationcited by 106
  34. 2020Weakly Supervised 3D Human Pose and Shape Reconstruction with Normalizing Flowscited by 94
  35. 2017Multi-Context Attention for Human Pose Estimationcited by 66
  36. 20203D Human Mesh Regression with Dense Correspondencecited by 66
  37. 2017Learning Feature Pyramids for Human Pose Estimationcited by 57
  38. 2020Differentiable Hierarchical Graph Grouping for Multi-Person Pose Estimationcited by 52

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 11; 2023: 30; 2024: 27; 2025: 21; 2026: 10.

99 papers.

Sort:

By area:

  1. 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 368
  2. 2023Effective Whole-body Pose Estimation with Two-stages Distillationcited by 307
  3. 2023BiFormer: Vision Transformer with Bi-Level Routing Attentioncited by 150
  4. 2023Beyond Appearance: a Semantic Controllable Self-Supervised Learning Framework for Human-Centric Visual Taskscited by 94
  5. 2024Model Tells You Where to Merge: Adaptive KV Cache Merging for LLMs on Long-Context Taskscited by 94
  6. 2025FastVGGT: Training-Free Acceleration of Visual Geometry Transformercited by 93
  7. 2023Joint Token Pruning and Squeezing Towards More Aggressive Compression of Vision Transformerscited by 79
  8. 2023DiffRate : Differentiable Compression Rate for Efficient Vision Transformerscited by 78
  9. 2023Which Tokens to Use? Investigating Token Reduction in Vision Transformerscited by 59
  10. 2023Video-Text as Game Players: Hierarchical Banzhaf Interaction for Cross-Modal Representation Learningcited by 45
  11. 2022Vision Transformer with Super Token Samplingcited by 43
  12. 2022TORE: Token Reduction for Efficient Human Mesh Recovery with Transformercited by 41
38 more
  1. 2023TF-CLIP: Learning Text-free CLIP for Video-based Person Re-Identificationcited by 40
  2. 2022G2P-DDM: Generating Sign Pose Sequence from Gloss Sequence with Discrete Diffusion Modelcited by 38
  3. 2022Pose for Everything: Towards Category-Agnostic Pose Estimationcited by 36
  4. 2023POTTER: Pooling Attention Transformer for Efficient Human Mesh Recoverycited by 31
  5. 2025DFormerv2: Geometry Self-Attention for RGBD Semantic Segmentationcited by 30
  6. 2025Visual Instance-aware Prompt Tuningcited by 29
  7. 2024ALGM: Adaptive Local-then-Global Token Merging for Efficient Semantic Segmentation with Plain Vision Transformerscited by 28
  8. 2023Hulk: A Universal Knowledge Translator for Human-Centric Taskscited by 22
  9. 2023DAT++: Spatially Dynamic Vision Transformer with Deformable Attentioncited by 21
  10. 2024Learning to Merge Tokens via Decoupled Embedding for Efficient Vision Transformerscited by 21
  11. 2024Layer- and Timestep-Adaptive Differentiable Token Compression Ratios for Efficient Diffusion Transformerscited by 21
  12. 2024Importance-Based Token Merging for Efficient Image and Video Generationcited by 20
  13. 2023MonoATT: Online Monocular 3D Object Detection with Adaptive Token Transformercited by 18
  14. 2024ScanFormer: Referring Expression Comprehension by Iteratively Scanningcited by 18
  15. 2024Token Cropr: Faster ViTs for Quite a Few Taskscited by 18
  16. 2023Mutual Information-Based Temporal Difference Learning for Human Pose Estimation in Videocited by 17
  17. 2022Learning Hierarchical Image Segmentation For Recognition and By Recognitioncited by 16
  18. 2023SpeechFormer++: A Hierarchical Efficient Framework for Paralinguistic Speech Processingcited by 16
  19. 2023Sparsifiner: Learning Sparse Instance-Dependent Attention for Efficient Vision Transformerscited by 16
  20. 2023Hourglass Tokenizer for Efficient Transformer-Based 3D Human Pose Estimationcited by 15
  21. 2025CATANet: Efficient Content-Aware Token Aggregation for Lightweight Image Super-Resolutioncited by 15
  22. 2025AVGGT: Rethinking Global Attention for Accelerating VGGTcited by 15
  23. 2022PoseTrans: A Simple Yet Effective Pose Transformation Augmentation for Human Pose Estimationcited by 14
  24. 2023Markerless human pose estimation for biomedical applications: a surveycited by 13
  25. 2024Agglomerative Token Clusteringcited by 13
  26. 2023Visual Dependency Transformers: Dependency Tree Emerges from Reversed Attentioncited by 12
  27. 2024Spatial-Semantic Collaborative Cropping for User Generated Contentcited by 12
  28. 2024TCFormer: Visual Recognition via Token Clustering Transformercited by 12
  29. 2024Pose Magic: Efficient and Temporally Consistent Human Pose Estimation with a Hybrid Mamba-GCN Networkcited by 12
  30. 2023AiluRus: A Scalable ViT Framework for Dense Predictioncited by 10
  31. 2024When Pedestrian Detection Meets Multi-Modal Learning: Generalist Model and Benchmark Datasetcited by 10
  32. 2026OneVision-Encoder: Codec-Aligned Sparsity as a Foundational Principle for Multimodal Intelligencecited by 10
  33. 2022FeatER: An Efficient Network for Human Reconstruction via Feature Map-Based TransformERcited by 9
  34. 2023You Only Learn One Query: Learning Unified Human Query for Single-Stage Multi-Person Multi-Task Human-Centric Perceptioncited by 9
  35. 2024Leveraging Temporal Contextualization for Video Action Recognitioncited by 9
  36. 2023All Keypoints You Need: Detecting Arbitrary Keypoints on the Body of Triple, High, and Long Jump Athletescited by 8
  37. 2023UMIFormer: Mining the Correlations between Similar Tokens for Multi-View 3D Reconstructioncited by 7
  38. 2023Extract-and-Adaptation Network for 3D Interacting Hand Mesh Recoverycited by 7

50 most cited of 99 All 99 (JSON)