mlsanityData to 14 Oct 2026

Towards Precise 3D Human Pose Estimation with Multi-Perspective Spatial-Temporal Relational Transformers

arXiv 2401.16700 ↗First posted 30 Jan 2024cs.CV cs.RO eess.IVcited by 1

Not on the map. Why?

Abstract (arXiv)

3D human pose estimation captures the human joint points in three-dimensional space while keeping the depth information and physical structure. That is essential for applications that require precise pose information, such as human-computer interaction, scene understanding, and rehabilitation training. Due to the challenges in data collection, mainstream datasets of 3D human pose estimation are primarily composed of multi-view video data collected in laboratory environments, which contains rich spatial-temporal correlation information besides the image frame content. Given the remarkable self-attention mechanism of transformers, capable of capturing the spatial-temporal correlation from multi-view video datasets, we propose a multi-stage framework for 3D sequence-to-sequence (seq2seq) human pose detection. Firstly, the spatial module represents the human pose feature by intra-image content, while the frame-image relation module extracts temporal relationships and 3D spatial positional relationship features between the multi-perspective images. Secondly, the self-attention mechanism is adopted to eliminate the interference from non-human body parts and reduce computing resources. Our method is evaluated on Human3.6M, a popular 3D human pose detection dataset. Experimental results demonstrate that our approach achieves state-of-the-art performance on this dataset. The source code will be available at https://github.com/WUJINHUAN/3D-human-pose.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 41 of its 49 references that are in the map. First posted on arXiv 30 Jan 2024.

Builds on

41 of 49 references.

References · 41

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2020End-to-End Object Detection with Transformerscited by 7,414
  6. 2017Mask R-CNNcited by 7,277
  7. 2016Layer Normalizationcited by 5,669
  8. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  9. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  10. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,064
  11. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 915
  12. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
29 more
  1. 2018Simple Baselines for Human Pose Estimation and Trackingcited by 502
  2. 2016Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervisioncited by 494
  3. 2022ViTPose: Simple Vision Transformer Baselines for Human Pose Estimationcited by 456
  4. 20183D human pose estimation in video with temporal convolutions and semi-supervised trainingcited by 403
  5. 2017A simple yet effective baseline for 3d human pose estimationcited by 401
  6. 2017Cascaded Pyramid Network for Multi-Person Pose Estimationcited by 341
  7. 2016Convolutional Pose Machinescited by 325
  8. 20213D Human Pose Estimation with Spatial and Temporal Transformerscited by 272
  9. 2016Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Posecited by 221
  10. 2020Deep Learning-Based Human Pose Estimation: A Surveycited by 207
  11. 2021MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimationcited by 151
  12. 2020TransPose: Keypoint Localization via Transformercited by 126
  13. 2018Ordinal Depth Supervision for 3D Human Pose Estimationcited by 107
  14. 2021TokenPose: Learning Keypoint Tokens for Human Pose Estimationcited by 107
  15. 2022Not All Tokens Are Equal: Human-centric Visual Analysis via Token Clustering Transformercited by 101
  16. 2021Pose Recognition with Cascade Transformerscited by 99
  17. 2020SRNet: Improving Generalization in 3D Human Pose Estimation with a Split-and-Recombine Approachcited by 94
  18. 2021Exploiting Temporal Contexts with Strided Transformer for 3D Human Pose Estimationcited by 94
  19. 2019Cross View Fusion for 3D Human Pose Estimationcited by 90
  20. 2017Exploiting temporal information for 3D pose estimationcited by 89
  21. 2020Motion Guided 3D Pose Estimation from Videoscited by 85
  22. 2019Rethinking on Multi-Stage Networks for Human Pose Estimationcited by 58
  23. 2019Chirality Nets for Human Pose Regressioncited by 31
  24. 2019Trajectory Space Factorization for Deep Video-Based 3D Human Pose Estimationcited by 30
  25. 2022Efficient Human Pose Estimation via 3D Event Point Cloudcited by 25
  26. 2021Trans4Trans: Efficient Transformer for Transparent Object and Semantic Scene Segmentation in Real-World Navigation Assistancecited by 22
  27. 2023FishDreamer: Towards Fisheye Semantic Completion via Unified Image Outpainting and Segmentationcited by 6
  28. 2022Beyond the Field-of-View: Enhancing Scene Visibility and Perception with Clip-Recurrent Transformercited by 5
  29. 2023Exploring Event-based Human Pose Estimation with 3D Event Representationscited by 4

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Human Motion Prediction via Test-domain-aware Adaptation with Easily-available Human Motions Estimated from Videoscited by 0