mlsanityData to 7 Oct 2026

ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation

arXiv 2204.12484 ↗First posted 26 Apr 2022cs.CVcited by 450

Not on the map. Why?

Cited by 22 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Although no specific domain knowledge is considered in the design, plain vision transformers have shown excellent performance in visual recognition tasks. However, little effort has been made to reveal the potential of such simple structures for pose estimation tasks. In this paper, we show the surprisingly good capabilities of plain vision transformers for pose estimation from various aspects, namely simplicity in model structure, scalability in model size, flexibility in training paradigm, and transferability of knowledge between models, through a simple baseline model called ViTPose. Specifically, ViTPose employs plain and non-hierarchical vision transformers as backbones to extract features for a given person instance and a lightweight decoder for pose estimation. It can be scaled up from 100M to 1B parameters by taking the advantages of the scalable model capacity and high parallelism of transformers, setting a new Pareto front between throughput and performance. Besides, ViTPose is very flexible regarding the attention type, input resolution, pre-training and finetuning strategy, as well as dealing with multiple pose tasks. We also empirically demonstrate that the knowledge of large ViTPose models can be easily transferred to small ones via a simple knowledge token. Experimental results show that our basic ViTPose model outperforms representative methods on the challenging MS COCO Keypoint Detection benchmark, while the largest model sets a new state-of-the-art. The code and models are available at https://github.com/ViTAE-Transformer/ViTPose.

Key facts. Cited by 450 papers in the map (data through 7 Oct 2026). Builds on 48 of its 62 references that are in the map. First posted on arXiv 26 Apr 2022.

Builds on

48 of 62 references.

References · 48

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  6. 2020End-to-End Object Detection with Transformerscited by 7,384
  7. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  8. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  9. 2018YOLOv3: An Incremental Improvementcited by 2,652
  10. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  11. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  12. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
36 more
  1. 2020Knowledge Distillation: A Surveycited by 1,578
  2. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  3. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  4. 2019On the Convergence of Adam and Beyondcited by 834
  5. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 723
  6. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 516
  7. 2018Simple Baselines for Human Pose Estimation and Trackingcited by 502
  8. 2013DeepPose: Human Pose Estimation via Deep Neural Networkscited by 305
  9. 2021ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Biascited by 216
  10. 2018CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmarkcited by 214
  11. 2019Distribution-Aware Coordinate Representation for Human Pose Estimationcited by 165
  12. 2021HRFormer: High-Resolution Transformer for Dense Predictioncited by 149
  13. 2021CrossFormer: A Versatile Vision Transformer Hinging on Cross-scale Attentioncited by 139
  14. 2020TransPose: Keypoint Localization via Transformercited by 126
  15. 2021AP-10K: A Benchmark for Animal Pose Estimation in the Wildcited by 123
  16. 2022ViTAEv2: Vision Transformer Advanced by Exploring Inductive Bias for Image Recognition and Beyondcited by 117
  17. 2020Empowering Things with Intelligence: A Survey of the Progress, Challenges, and Opportunities in Artificial Intelligence of Thingscited by 110
  18. 2019Cross-Domain Adaptation for Animal Pose Estimationcited by 108
  19. 2018Pose2Seg: Detection Free Human Instance Segmentationcited by 107
  20. 2021TokenPose: Learning Keypoint Tokens for Human Pose Estimationcited by 106
  21. 2021Pose Recognition with Cascade Transformerscited by 99
  22. 2021Scalable Vision Transformers with Hierarchical Poolingcited by 84
  23. 2019The Devil is in the Details: Delving into Unbiased Data Processing for Human Pose Estimationcited by 79
  24. 2017AI Challenger : A Large-scale Dataset for Going Deeper in Image Understandingcited by 73
  25. 2020Learning Delicate Local Representations for Multi-Person Pose Estimationcited by 65
  26. 2019Rethinking on Multi-Stage Networks for Human Pose Estimationcited by 58
  27. 2022VSA: Learning Varied-Size Window Attention in Vision Transformerscited by 43
  28. 2022BigDetection: A Large-scale Benchmark for Improved Object Detector Pre-trainingcited by 33
  29. 2021Multi-Instance Pose Networks: Rethinking Top-Down Pose Estimationcited by 30
  30. 2021Scaled ReLU Matters for Training Vision Transformerscited by 29
  31. 2020Peeking into occluded joints: A novel framework for crowd pose estimationcited by 25
  32. 2020Adversarial Semantic Data Augmentation for Human Pose Estimationcited by 22
  33. 2020Towards High Performance Human Keypoint Detectioncited by 20
  34. 2021ELSA: Enhanced Local Self-Attention for Vision Transformercited by 18
  35. 2019Cascade Feature Aggregation for Human Pose Estimationcited by 8
  36. 2017An Implementation of Faster RCNN with Study for Region Samplingcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 15; 2023: 101; 2024: 116; 2025: 109; 2026: 109.

450 papers.

Sort:

By area:

  1. 2023Humans in 4D: Reconstructing and Tracking Humans with Transformerscited by 344
  2. 2023Reconstructing Hands in 3D with Transformerscited by 298
  3. 2024Sapiens: Foundation for Human Vision Modelscited by 229
  4. 2023Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Datasetcited by 220
  5. 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 205
  6. 2023WHAM: Reconstructing World-grounded Humans with Accurate 3D Motioncited by 200
  7. 2023RTMPose: Real-Time Multi-Person Pose Estimation based on MMPosecited by 170
  8. 2022CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flowcited by 165
  9. 2023InstructDiffusion: A Generalist Modeling Interface for Vision Taskscited by 162
  10. 2023SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimationcited by 154
  11. 2022Advancing Plain Vision Transformer Towards Remote Sensing Foundation Modelcited by 149
  12. 2023Decoupling Human and Camera Motion from Videos in the Wildcited by 140
38 more
  1. 2023One-Stage 3D Whole-Body Mesh Recovery with Component Aware Transformercited by 128
  2. 2024WiLoR: End-to-end 3D Hand Localization and Reconstruction in-the-wildcited by 117
  3. 2024TRAM: Global Trajectory and Motion of 3D Humans from in-the-wild Videoscited by 105
  4. 2024Multi-HMR: Multi-Person Whole-Body Human Mesh Recovery in a Single Shotcited by 98
  5. 2024TokenHMR: Advancing Human Mesh Recovery with a Tokenized Pose Representationcited by 93
  6. 2023TRACE: 5D Temporal Regression of Avatars with Dynamic Cameras in 3D Environmentscited by 89
  7. 2024DINO-X: A Unified Vision Model for Open-World Object Detection and Understandingcited by 85
  8. 2025Wan-Animate: Unified Character Animation and Replacement with Holistic Replicationcited by 76
  9. 2024What Matters When Repurposing Diffusion Models for General Dense Perception Tasks?cited by 74
  10. 2023ChatPose: Chatting about 3D Human Posecited by 71
  11. 2023Human-Art: A Versatile Human-Centric Dataset Bridging Natural and Artificial Scenescited by 68
  12. 2023HyperHuman: Hyper-Realistic Human Generation with Latent Structural Diffusioncited by 68
  13. 2022ViTPose++: Vision Transformer for Generic Body Pose Estimationcited by 65
  14. 2024CameraHMR: Aligning People with Perspectivecited by 64
  15. 2025Wan-S2V: Audio-Driven Cinematic Video Generationcited by 60
  16. 2023UniHCP: A Unified Model for Human-Centric Perceptionscited by 59
  17. 2025Visual Imitation Enables Contextual Humanoid Controlcited by 57
  18. 2023ViTMatte: Boosting Image Matting with Pretrained Plain Vision Transformerscited by 56
  19. 2025ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learningcited by 56
  20. 2022APT-36K: A Large-scale Benchmark for Animal Pose Estimation and Trackingcited by 55
  21. 2023DECO: Dense Estimation of 3D Human-Scene Contact In The Wildcited by 54
  22. 2024HOI-M3:Capture Multiple Humans and Objects Interaction within Contextual Environmentcited by 52
  23. 2023Exploring Plain ViT Reconstruction for Multi-class Unsupervised Anomaly Detectioncited by 47
  24. 2026Ultralytics YOLO26: Unified Real-Time End-to-End Vision Modelscited by 47
  25. 2023Vision Transformer with Quadrangle Attentioncited by 46
  26. 2023Human Pose as Compositional Tokenscited by 45
  27. 2024nnMamba: 3D Biomedical Image Segmentation, Classification and Landmark Detection with State Space Modelcited by 45
  28. 2024Hamba: Single-view 3D Hand Reconstruction with Graph-guided Bi-Scanning Mambacited by 45
  29. 2023Generative Proxemics: A Prior for 3D Social Interaction from Imagescited by 43
  30. 2023ViPLO: Vision Transformer based Pose-Conditioned Self-Loop Graph for Human-Object Interaction Detectioncited by 41
  31. 2023HumanBench: Towards General Human-centric Perception with Projector Assisted Pretrainingcited by 38
  32. 2023MAS: Multi-view Ancestral Sampling for 3D motion generation using 2D diffusioncited by 37
  33. 2025DICEPTION: A Generalist Diffusion Model for Visual Perceptual Taskscited by 37
  34. 2023DiffPose: SpatioTemporal Diffusion Model for Video-Based Human Pose Estimationcited by 36
  35. 2025Human3R: Everyone Everywhere All at Oncecited by 35
  36. 2022Expediting Large-Scale Vision Transformer for Dense Prediction without Fine-tuningcited by 33
  37. 2023PACE: Human and Camera Motion Estimation from in-the-wild Videoscited by 33
  38. 2023HAP: Structure-Aware Masked Image Modeling for Human-Centric Perceptioncited by 33

50 most cited of 450 All 450 (JSON)