mlsanityData to 14 Oct 2026

Cross-Attention of Disentangled Modalities for 3D Human Mesh Recovery with Transformers

arXiv 2207.13820 ↗First posted 27 Jul 2022cs.CV cs.AI cs.LGcited by 81

Not on the map. Why?

Abstract (arXiv)

Transformer encoder architectures have recently achieved state-of-the-art results on monocular 3D human mesh reconstruction, but they require a substantial number of parameters and expensive computations. Due to the large memory overhead and slow inference speed, it is difficult to deploy such models for practical use. In this paper, we propose a novel transformer encoder-decoder architecture for 3D human mesh reconstruction from a single image, called FastMETRO. We identify the performance bottleneck in the encoder-based transformers is caused by the token design which introduces high complexity interactions among input tokens. We disentangle the interactions via an encoder-decoder architecture, which allows our model to demand much fewer parameters and shorter inference time. In addition, we impose the prior knowledge of human body's morphological relationship via attention masking and mesh upsampling operations, which leads to faster convergence with higher accuracy. Our FastMETRO improves the Pareto-front of accuracy and efficiency, and clearly outperforms image-based methods on Human3.6M and 3DPW. Furthermore, we validate its generalizability on FreiHAND.

Key facts. Cited by 81 papers in the map (data through 14 Oct 2026). Builds on 38 of its 50 references that are in the map. First posted on arXiv 27 Jul 2022.

Builds on

38 of 50 references.

References · 38

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  6. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  7. 2020End-to-End Object Detection with Transformerscited by 7,414
  8. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  9. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,420
  10. 2017End-to-end Recovery of Human Shape and Posecited by 885
  11. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
  12. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 695
26 more
  1. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 627
  2. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 620
  3. 2020End-to-End Video Instance Segmentation with Transformerscited by 443
  4. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 418
  5. 2021PARE: Part Attention Regressor for 3D Human Body Estimationcited by 337
  6. 2020I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Imagecited by 274
  7. 20213D Human Pose Estimation with Spatial and Temporal Transformerscited by 272
  8. 2019FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Imagescited by 271
  9. 2019Convolutional Mesh Regression for Single-Image Human Shape Reconstructioncited by 270
  10. 2018Generating 3D faces using Convolutional Mesh Autoencoderscited by 268
  11. 2021Mesh Graphormercited by 265
  12. 2021PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loopcited by 250
  13. 2020Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Posecited by 248
  14. 2018Learning to Estimate 3D Human Pose and Shape from a Single Color Imagecited by 226
  15. 2020Monocular, One-stage, Regression of Multiple 3D Peoplecited by 218
  16. 2017Unite the People: Closing the Loop Between 3D and 2D Human Representationscited by 205
  17. 2017Single-Shot Multi-Person 3D Pose Estimation From Monocular RGBcited by 195
  18. 2021HOTR: End-to-End Human-Object Interaction Detection with Transformerscited by 183
  19. 2020Coherent Reconstruction of Multiple Humans from a Single Imagecited by 136
  20. 2020Normalized and Geometry-Aware Self-Attention Network for Image Captioningcited by 63
  21. 2021Paint Transformer: Feed Forward Neural Painting with Stroke Predictioncited by 62
  22. 2021Learning to Regress Bodies from Images using Differentiable Semantic Renderingcited by 39
  23. 2017MonoCap: Monocular Human Motion Capture using a CNN Coupled with a Geometric Priorcited by 36
  24. 2018A Mixed Classification-Regression Framework for 3D Pose Estimation from 2D Imagescited by 19
  25. 2022Collaborative Transformers for Grounded Situation Recognitioncited by 19
  26. 2021Grounded Situation Recognition with Transformerscited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 4; 2023: 28; 2024: 32; 2025: 11; 2026: 6.

81 papers.

Sort:

By area:

  1. 2023Humans in 4D: Reconstructing and Tracking Humans with Transformerscited by 350
  2. 2022MotionBERT: A Unified Perspective on Learning Human Motion Representationscited by 193
  3. 2024TRAM: Global Trajectory and Motion of 3D Humans from in-the-wild Videoscited by 106
  4. 2023EMDB: The Electromagnetic Database of Global 3D Human Pose and Shape in the Wildcited by 97
  5. 2023Paint-it: Text-to-Texture Synthesis via Deep Convolutional Texture Map Optimization and Physically-Based Renderingcited by 84
  6. 2024Neural Localizer Fields for Continuous 3D Human Pose and Shape Estimationcited by 59
  7. 2024RoHM: Robust Human Motion Reconstruction via Diffusioncited by 48
  8. 2024Hamba: Single-view 3D Hand Reconstruction with Graph-guided Bi-Scanning Mambacited by 45
  9. 20233D Human Mesh Estimation from Virtual Markerscited by 44
  10. 2024AiOS: All-in-One-Stage Expressive Human Pose and Shape Estimationcited by 42
  11. 2022TORE: Token Reduction for Efficient Human Mesh Recovery with Transformercited by 41
  12. 2023Zolly: Zoom Focal Length Correctly for Perspective-Distorted Human Mesh Reconstructioncited by 41
38 more
  1. 2024Score-Guided Diffusion for 3D Human Recoverycited by 39
  2. 2023Probabilistic Human Mesh Recovery in 3D Scenes from Egocentric Viewscited by 33
  3. 2023POTTER: Pooling Attention Transformer for Efficient Human Mesh Recoverycited by 31
  4. 2023Distribution-Aligned Diffusion for Human Mesh Recoverycited by 27
  5. 2024A Simple Baseline for Efficient Hand Mesh Reconstructioncited by 27
  6. 2023Co-Evolution of Pose and Mesh for 3D Human Body Estimation from Videocited by 25
  7. 2023Sampling is Matter: Point-guided 3D Human Mesh Reconstructioncited by 24
  8. 2023CVTHead: One-shot Controllable Head Avatar with Vertex-feature Transformercited by 24
  9. 2023Hulk: A Universal Knowledge Translator for Human-Centric Taskscited by 22
  10. 2023Reconstructing Interacting Hands with Interaction Prior from Monocular Imagescited by 18
  11. 2024SAT-HMR: Real-Time Multi-Person 3D Mesh Estimation via Scale-Adaptive Tokenscited by 16
  12. 2025BioPose: Biomechanically-accurate 3D Pose Estimation from Monocular Videoscited by 16
  13. 2024Complementing Event Streams and RGB Frames for Hand Mesh Reconstructioncited by 14
  14. 2024MEGA: Masked Generative Autoencoder for Human Mesh Recoverycited by 14
  15. 2025WorldPose: A World Cup Dataset for Global 3D Human Pose Estimationcited by 14
  16. 2023VQ-HPS: Human Pose and Shape Estimation in a Vector-Quantized Latent Spacecited by 13
  17. 2022MPT: Mesh Pre-Training with Transformers for Human Pose and Mesh Reconstructioncited by 11
  18. 2024Deep learning for 3D human pose estimation and mesh recovery: A surveycited by 11
  19. 2024GST: Precise 3D Human Body from a Single Image with Gaussian Splatting Transformerscited by 11
  20. 2023Spectral Graphormer: Spectral Graph-based Transformer for Egocentric Two-Hand Reconstruction using Multi-View Color Imagescited by 10
  21. 2024DICE: End-to-end Deformation Capture of Hand-Face Interactions from a Single Imagecited by 9
  22. 2024HandOS: 3D Hand Reconstruction in One Stagecited by 9
  23. 2023Decoupled Iterative Refinement Framework for Interacting Hands Reconstruction from a Single RGB Imagecited by 8
  24. 2024MeshPose: Unifying DensePose and 3D Body Mesh reconstructioncited by 8
  25. 2025End-to-End HOI Reconstruction Transformer with Graph-based Encodingcited by 8
  26. 2025ODHSR: Online Dense 3D Reconstruction of Humans and Scenes from Monocular Videoscited by 8
  27. 2023Learning Dense UV Completion for Human Mesh Recoverycited by 7
  28. 2024Human Mesh Recovery from Arbitrary Multi-view Imagescited by 6
  29. 2024HumMUSS: Human Motion Understanding using State Space Modelscited by 6
  30. 2023PHRIT: Parametric Hand Representation with Implicit Templatecited by 5
  31. 2023CLIP-Hand3D: Exploiting 3D Hand Pose Estimation via Context-Aware Promptingcited by 5
  32. 2023A Large-Scale 3D Face Mesh Video Dataset via Neural Re-parameterized Optimizationcited by 5
  33. 2024MaskHand: Generative Masked Modeling for Robust Hand Mesh Reconstruction in the Wildcited by 5
  34. 2023ReBaR: Reference-Based Reasoning for Robust Pose Estimation from Monocular Imagescited by 4
  35. 2023GraMMaR: Ground-aware Motion Model for 3D Human Motion Reconstructioncited by 4
  36. 2025FastHMR: Accelerating Human Mesh Recovery via Token and Layer Merging with Diffusion Decodingcited by 4
  37. 2026Fast SAM 3D Body: Accelerating SAM 3D Body for Real-Time Full-Body Human Mesh Recoverycited by 4
  38. 2023Which One? Leveraging Context Between Objects and Multiple Views for Language Groundingcited by 3

50 most cited of 81 All 81 (JSON)