mlsanityData to 14 Oct 2026

Efficient Vision Transformer for Human Pose Estimation via Patch Selection

arXiv 2306.04225 ↗First posted 7 Jun 2023cs.CV cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

While Convolutional Neural Networks (CNNs) have been widely successful in 2D human pose estimation, Vision Transformers (ViTs) have emerged as a promising alternative to CNNs, boosting state-of-the-art performance. However, the quadratic computational complexity of ViTs has limited their applicability for processing high-resolution images. In this paper, we propose three methods for reducing ViT's computational complexity, which are based on selecting and processing a small number of most informative patches while disregarding others. The first two methods leverage a lightweight pose estimation network to guide the patch selection process, while the third method utilizes a set of learnable joint tokens to ensure that the selected patches contain the most important information about body joints. Experiments across six benchmarks show that our proposed methods achieve a significant reduction in computational complexity, ranging from 30% to 44%, with only a minimal drop in accuracy between 0% and 3.5%.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 33 of its 46 references that are in the map. First posted on arXiv 7 Jun 2023.

Builds on

33 of 46 references.

References · 33

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  4. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  5. 2019Generating Long Sequences with Sparse Transformerscited by 1,546
  6. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  7. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,209
  8. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,200
  9. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,162
  10. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 915
  11. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
  12. 2018Simple Baselines for Human Pose Estimation and Trackingcited by 502
21 more
  1. 2020Efficient Content-Based Sparse Attention with Routing Transformerscited by 471
  2. 2022ViTPose: Simple Vision Transformer Baselines for Human Pose Estimationcited by 456
  3. 2022Not All Patches are What You Need: Expediting Vision Transformers via Token Reorganizationscited by 363
  4. 2016Convolutional Pose Machinescited by 325
  5. 2013DeepPose: Human Pose Estimation via Deep Neural Networkscited by 305
  6. 2019HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimationcited by 247
  7. 2021AdaViT: Adaptive Vision Transformers for Efficient Image Recognitioncited by 228
  8. 2018CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmarkcited by 214
  9. 2016Associative Embedding: End-to-End Learning for Joint Detection and Groupingcited by 212
  10. 2021Adaptive Token Sampling For Efficient Vision Transformerscited by 183
  11. 2021HRFormer: High-Resolution Transformer for Dense Predictioncited by 149
  12. 2021TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?cited by 127
  13. 2020TransPose: Keypoint Localization via Transformercited by 126
  14. 2018Pose2Seg: Detection Free Human Instance Segmentationcited by 107
  15. 2021TokenPose: Learning Keypoint Tokens for Human Pose Estimationcited by 107
  16. 2021Lite-HRNet: A Lightweight High-Resolution Networkcited by 91
  17. 2021Scalable Vision Transformers with Hierarchical Poolingcited by 84
  18. 2017AI Challenger : A Large-scale Dataset for Going Deeper in Image Understandingcited by 73
  19. 2021Token Pooling in Vision Transformerscited by 69
  20. 2022JRDB-Pose: A Large-scale Dataset for Multi-Person Pose Estimation and Trackingcited by 25
  21. 2020PoWER-BERT: Accelerating BERT Inference via Progressive Word-vector Eliminationcited by 10

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 3.

4 papers.

  1. 2024Enhancing Efficiency in Vision Transformer Networks: Design Techniques and Insightscited by 4
  2. 2025When Less is Enough: Adaptive Token Reduction for Efficient Image Representationcited by 2
  3. 2025Computerized Assessment of Motor Imitation for Distinguishing Autism in Video (CAMI-2DNet)cited by 1
  4. 2025Transformers with Joint Tokens and Local-Global Attention for Efficient Human Pose Estimationcited by 0