mlsanityData to 7 Oct 2026

End-to-End Human Pose and Mesh Reconstruction with Transformers

arXiv 2012.09760 ↗First posted 17 Dec 2020cs.CVcited by 417

Not on the map. Why?

Abstract (arXiv)

We present a new method, called MEsh TRansfOrmer (METRO), to reconstruct 3D human pose and mesh vertices from a single image. Our method uses a transformer encoder to jointly model vertex-vertex and vertex-joint interactions, and outputs 3D joint coordinates and mesh vertices simultaneously. Compared to existing techniques that regress pose and shape parameters, METRO does not rely on any parametric mesh models like SMPL, thus it can be easily extended to other objects such as hands. We further relax the mesh topology and allow the transformer self-attention mechanism to freely attend between any two vertices, making it possible to learn non-local relationships among mesh vertices and joints. With the proposed masked vertex modeling, our method is more robust and effective in handling challenging situations like partial occlusions. METRO generates new state-of-the-art results for human mesh reconstruction on the public Human3.6M and 3DPW datasets. Moreover, we demonstrate the generalizability of METRO to 3D hand reconstruction in the wild, outperforming existing state-of-the-art methods on FreiHAND dataset. Code and pre-trained models are available at https://github.com/microsoft/MeshTransformer.

Key facts. Cited by 417 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 49; 2022: 96; 2023: 106; 2024: 85; 2025: 40; 2026: 41.

417 papers.

Sort:

By area:

  1. 2021Transformers in Vision: A Surveycited by 1,009
  2. 2021Multiscale Vision Transformerscited by 896
  3. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 647
  4. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 477
  5. 2022Multimodal Learning with Transformers: A Surveycited by 399
  6. 2023Humans in 4D: Reconstructing and Tracking Humans with Transformerscited by 344
  7. 2023Reconstructing Hands in 3D with Transformerscited by 298
  8. 20213D Human Pose Estimation with Spatial and Temporal Transformerscited by 271
  9. 2021Mesh Graphormercited by 264
  10. 2021PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loopcited by 250
  11. 2022NeuMan: Neural Human Radiance Field from a Single Videocited by 244
  12. 2022CLIFF: Carrying Location Information in Full Frames into Human Pose and Shape Estimationcited by 237
38 more
  1. 2023BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motioncited by 229
  2. 2021FaceFormer: Speech-Driven 3D Facial Animation with Transformerscited by 208
  3. 2022MotionBERT: A Unified Perspective on Learning Human Motion Representationscited by 188
  4. 2022Capturing and Inferring Dense Full-Body Human-Scene Contactcited by 176
  5. 2021FrankMocap: A Monocular 3D Whole-Body Pose Estimation System via Regression and Integrationcited by 168
  6. 2023FastViT: A Fast Hybrid Vision Transformer using Structural Reparameterizationcited by 161
  7. 2022MixSTE: Seq2seq Mixed Spatio-Temporal Encoder for 3D Human Pose Estimation in Videocited by 158
  8. 2022PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Imagescited by 152
  9. 2021MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimationcited by 151
  10. 2021GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Camerascited by 151
  11. 2021Putting People in their Place: Monocular Regression of 3D People in Depthcited by 141
  12. 2021Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Timecited by 140
  13. 2021Collaborative Regression of Expressive Bodies using Moderationcited by 137
  14. 2021EgoBody: Human Body Shape and Motion of Interacting People from Head-Mounted Devicescited by 136
  15. 2022Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis: From Theory to Practicecited by 136
  16. 2021Keypoint Transformer: Solving Joint Identification in Challenging Hands and Object Interactions for Accurate 3D Pose Estimationcited by 122
  17. 2024WiLoR: End-to-end 3D Hand Localization and Reconstruction in-the-wildcited by 117
  18. 2022Is Attention All That NeRF Needs?cited by 113
  19. 2022AvatarPoser: Articulated Full-Body Pose Tracking from Sparse Motion Sensingcited by 107
  20. 2021TokenPose: Learning Keypoint Tokens for Human Pose Estimationcited by 106
  21. 2024TRAM: Global Trajectory and Motion of 3D Humans from in-the-wild Videoscited by 105
  22. 2022Recovering 3D Human Mesh from Monocular Images: A Surveycited by 103
  23. 20233D Human Pose Estimation via Intuitive Physicscited by 100
  24. 2022Not All Tokens Are Equal: Human-centric Visual Analysis via Token Clustering Transformercited by 99
  25. 2022M3ViT: Mixture-of-Experts Vision Transformer for Efficient Multi-task Learning with Model-Accelerator Co-designcited by 97
  26. 2022HandOccNet: Occlusion-Robust 3D Hand Mesh Estimation Networkcited by 95
  27. 2021Exploiting Temporal Contexts with Strided Transformer for 3D Human Pose Estimationcited by 94
  28. 2023TRACE: 5D Temporal Regression of Avatars with Dynamic Cameras in 3D Environmentscited by 89
  29. 2022Interacting Attention Graph for Single Image Two-Hand Reconstructioncited by 87
  30. 2024FMGS: Foundation Model Embedded 3D Gaussian Splatting for Holistic 3D Scene Understandingcited by 86
  31. 2024DINO-X: A Unified Vision Model for Open-World Object Detection and Understandingcited by 85
  32. 2022Cross-Attention of Disentangled Modalities for 3D Human Mesh Recovery with Transformerscited by 81
  33. 2022P-STMO: Pre-Trained Spatial Temporal Many-to-One Model for 3D Human Pose Estimationcited by 80
  34. 2024OAKINK2: A Dataset of Bimanual Hands-Object Manipulation in Complex Task Completioncited by 78
  35. 2023HOLD: Category-agnostic 3D Reconstruction of Interacting Hands and Objects from Videocited by 76
  36. 2021MobRecon: Mobile-Friendly Hand Mesh Reconstruction from Monocular Imagecited by 74
  37. 2023E^2VPT: An Effective and Efficient Approach for Visual Prompt Tuningcited by 70
  38. 2022Coarse-to-Fine Sparse Transformer for Hyperspectral Image Reconstructioncited by 67

50 most cited of 417 All 417 (JSON)