mlsanityData to 7 Oct 2026

Uplift and Upsample: Efficient 3D Human Pose Estimation with Uplifting Transformers

arXiv 2210.06110 ↗First posted 12 Oct 2022cs.CVcited by 33

Not on the map. Why?

Abstract (arXiv)

The state-of-the-art for monocular 3D human pose estimation in videos is dominated by the paradigm of 2D-to-3D pose uplifting. While the uplifting methods themselves are rather efficient, the true computational complexity depends on the per-frame 2D pose estimation. In this paper, we present a Transformer-based pose uplifting scheme that can operate on temporally sparse 2D pose sequences but still produce temporally dense 3D pose estimates. We show how masked token modeling can be utilized for temporal upsampling within Transformer blocks. This allows to decouple the sampling rate of input 2D poses and the target frame rate of the video and drastically decreases the total computational complexity. Additionally, we explore the option of pre-training on large motion capture archives, which has been largely neglected so far. We evaluate our method on two popular benchmark datasets: Human3.6M and MPI-INF-3DHP. With an MPJPE of 45.0 mm and 46.9 mm, respectively, our proposed method can compete with the state-of-the-art while reducing inference time by a factor of 12. This enables real-time throughput with variable consumer hardware in stationary and mobile applications. We release our code and models at https://github.com/goldbricklemon/uplift-upsample-3dhpe

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Builds on 44 of its 60 references that are in the map. First posted on arXiv 12 Oct 2022.

Builds on

44 of 60 references.

References · 44

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2020End-to-End Object Detection with Transformerscited by 7,384
  5. 2017Mask R-CNNcited by 7,259
  6. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  7. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
  8. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,447
  9. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  10. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,199
  11. 2016Deep Networks with Stochastic Depthcited by 973
  12. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
32 more
  1. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 625
  2. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 615
  3. 2016Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervisioncited by 493
  4. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 417
  5. 2017A simple yet effective baseline for 3d human pose estimationcited by 401
  6. 20183D human pose estimation in video with temporal convolutions and semi-supervised trainingcited by 401
  7. 2017Cascaded Pyramid Network for Multi-Person Pose Estimationcited by 341
  8. 2018Learning 3D Human Dynamics from Videocited by 315
  9. 20213D Human Pose Estimation with Spatial and Temporal Transformerscited by 271
  10. 2019HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimationcited by 247
  11. 2022MixSTE: Seq2seq Mixed Spatio-Temporal Encoder for 3D Human Pose Estimation in Videocited by 158
  12. 2021MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimationcited by 151
  13. 2017VNect: Real-time 3D Human Pose Estimation with a Single RGB Cameracited by 129
  14. 2021TokenPose: Learning Keypoint Tokens for Human Pose Estimationcited by 106
  15. 20182D/3D Pose Estimation and Action Recognition using Multitask Deep Learningcited by 103
  16. 2021Exploiting Temporal Contexts with Strided Transformer for 3D Human Pose Estimationcited by 94
  17. 2021PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimationcited by 88
  18. 2019RepNet: Weakly Supervised Training of an Adversarial Reprojection Network for 3D Human Pose Estimationcited by 85
  19. 2020Motion Guided 3D Pose Estimation from Videoscited by 85
  20. 2020Cascaded deep monocular 3D human pose estimation with evolutionary training datacited by 80
  21. 2022P-STMO: Pre-Trained Spatial Temporal Many-to-One Model for 3D Human Pose Estimationcited by 80
  22. 2018Unsupervised Geometry-Aware Representation for 3D Human Pose Estimationcited by 73
  23. 2019XNect: Real-time Multi-Person 3D Motion Capture with a Single RGB Cameracited by 68
  24. 2021Conditional Directed Graph Convolution for 3D Human Pose Estimationcited by 52
  25. 2018Can 3D Pose be Learned from 2D Projections Alone?cited by 44
  26. 2018Memory Attention Networks for Skeleton-based Action Recognitioncited by 36
  27. 2019Trajectory Space Factorization for Deep Video-Based 3D Human Pose Estimationcited by 30
  28. 2021Learning Multi-Granular Spatio-Temporal Graph Network for Skeleton-based Action Recognitioncited by 25
  29. 2022PoseTriplet: Co-evolving 3D Human Pose Estimation, Imitation, and Hallucination under Self-supervisioncited by 25
  30. 2021Improving Robustness and Accuracy via Relative Information Encoding in 3D Human Pose Estimationcited by 18
  31. 2020Lightweight 3D Human Pose Estimation Network Training Using Teacher-Student Learningcited by 7
  32. 2017Robust 3D Action Recognition through Sampling Local Appearances and Global Distributionscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 9; 2025: 15; 2026: 2.

33 papers.

Sort:

By area:

  1. 2023PoseFormerV2: Exploring Frequency Domain for Efficient and Robust 3D Human Pose Estimationcited by 91
  2. 2023MotionAGFormer: Enhancing 3D Human Pose Estimation with a Transformer-GCNFormer Networkcited by 62
  3. 20233D Human Pose Perception from Egocentric Stereo Videoscited by 23
  4. 2023Hourglass Tokenizer for Efficient Transformer-Based 3D Human Pose Estimationcited by 15
  5. 2025AthletePose3D: A Benchmark Dataset for 3D Human Pose Estimation and Kinematic Validation in Athletic Movementscited by 14
  6. 2024Pose Magic: Efficient and Temporally Consistent Human Pose Estimation with a Hybrid Mamba-GCN Networkcited by 12
  7. 2023Regular Splitting Graph Network for 3D Human Pose Estimationcited by 11
  8. 2023A Single 2D Pose with Context is Worth Hundreds for 3D Human Pose Estimationcited by 11
  9. 2025Bring Your Rear Cameras for Egocentric 3D Human Pose Estimationcited by 7
  10. 2024HumMUSS: Human Motion Understanding using State Space Modelscited by 6
  11. 2025H2OT: Hierarchical Hourglass Tokenizer for Efficient Video Pose Transformerscited by 5
  12. 2024EgoPoseFormer: A Simple Baseline for Stereo Egocentric 3D Human Pose Estimationcited by 4
21 more
  1. 2025HGMamba: Enhancing 3D Human Pose Estimation with a HyperGCN-Mamba Networkcited by 4
  2. 2025Towards Ball Spin and Trajectory Analysis in Table Tennis Broadcast Videos via Physically Grounded Synthetic-to-Real Transfercited by 4
  3. 2024Enhancing 3D Human Pose Estimation Amidst Severe Occlusion with Dual Transformer Fusioncited by 3
  4. 2025A Structure-aware and Motion-adaptive Framework for 3D Human Pose Estimation with Mambacited by 3
  5. 2024STGFormer: Spatio-Temporal GraphFormer for 3D Human Pose Estimation in Videocited by 2
  6. 2024Leveraging Anthropometric Measurements to Improve Human Mesh Estimation and Ensure Consistent Body Shapescited by 2
  7. 2025SCJD: Sparse Correlation and Joint Distillation for Efficient 3D Human Pose Estimationcited by 2
  8. 2025Mamba-Driven Topology Fusion for Monocular 3D Human Pose Estimationcited by 2
  9. 2023Efficient, Self-Supervised Human Pose Estimation with Inductive Prior Tuningcited by 1
  10. 2024Graph and Skipped Transformer: Exploiting Spatial and Temporal Modeling Capacities for Efficient 3D Human Pose Estimationcited by 1
  11. 2025Efficient 2D to Full 3D Human Pose Uplifting including Joint Rotationscited by 1
  12. 2025Spectral Compression Transformer with Line Pose Graph for Monocular 3D Human Pose Estimationcited by 1
  13. 2025PriorFormer: A Transformer for Real-time Monocular 3D Human Pose Estimation with Versatile Geometric Priorscited by 1
  14. 2025SasMamba: A Lightweight Structure-Aware Stride State Space Model for 3D Human Pose Estimationcited by 1
  15. 2026TT4D: A Pipeline and Dataset for Table Tennis 4D Reconstruction From Monocular Videoscited by 1
  16. 2024Learning Human Motion from Monocular Videos via Cross-Modal Manifold Alignmentcited by 0
  17. 2024VioPose: Violin Performance 4D Pose Estimation by Hierarchical Audiovisual Inferencecited by 0
  18. 2025HDiffTG: A Lightweight Hybrid Diffusion-Transformer-GCN Architecture for 3D Human Pose Estimationcited by 0
  19. 2025Efficient Diffusion-Based 3D Human Pose Estimation with Hierarchical Temporal Pruningcited by 0
  20. 2025Explainable Neural Inverse Kinematics for Obstacle-Aware Robotic Manipulation: A Comparative Analysis of IKNet Variantscited by 0
  21. 2026TSR-Ego: Temporally Guided Stereo Refinement Framework for Egocentric 3D Human Pose Estimationcited by 0