# Efficient 2D to Full 3D Human Pose Uplifting including Joint Rotations

arXiv: https://arxiv.org/abs/2504.09953 · HTML: https://mlsanity.com/paper/2504.09953/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 29 references that are in the map.
- First posted on arXiv 14 Apr 2025.

## Builds on (most cited first, 24 of 24)

- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [3D human pose estimation in video with temporal convolutions and semi-supervised training](https://mlsanity.com/paper/1811.11742/) (2018)
- [Humans in 4D: Reconstructing and Tracking Humans with Transformers](https://mlsanity.com/paper/2305.20091/) (2023)
- [3D Human Pose Estimation with Spatial and Temporal Transformers](https://mlsanity.com/paper/2103.10455/) (2021)
- [DeMoN: Depth and Motion Network for Learning Monocular Stereo](https://mlsanity.com/paper/1612.02401/) (2016)
- [Monocular Expressive Body Regression through Body-Driven Attention](https://mlsanity.com/paper/2008.09062/) (2020)
- [SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimation](https://mlsanity.com/paper/2309.17448/) (2023)
- [Putting People in their Place: Monocular Regression of 3D People in Depth](https://mlsanity.com/paper/2112.08274/) (2021)
- [Collaborative Regression of Expressive Bodies using Moderation](https://mlsanity.com/paper/2105.05301/) (2021)
- [Multi-HMR: Multi-Person Whole-Body Human Mesh Recovery in a Single Shot](https://mlsanity.com/paper/2402.14654/) (2024)
- [Exploiting Temporal Contexts with Strided Transformer for 3D Human Pose Estimation](https://mlsanity.com/paper/2103.14304/) (2021)
- [An Analysis of SVD for Deep Rotation Estimation](https://mlsanity.com/paper/2006.14616/) (2020)
- [Accurate 3D Hand Pose Estimation for Whole-Body 3D Human Mesh Estimation](https://mlsanity.com/paper/2011.11534/) (2020)
- [Learning to Estimate Robust 3D Human Mesh from In-the-Wild Crowded Scenes](https://mlsanity.com/paper/2104.07300/) (2021)
- [Conditional Directed Graph Convolution for 3D Human Pose Estimation](https://mlsanity.com/paper/2107.07797/) (2021)
- [Uplift and Upsample: Efficient 3D Human Pose Estimation with Uplifting Transformers](https://mlsanity.com/paper/2210.06110/) (2022)
- [PSVT: End-to-End Multi-person 3D Pose and Shape Estimation with Progressive Video Transformers](https://mlsanity.com/paper/2303.09187/) (2023)
- [Deep-6DPose: Recovering 6D Object Pose from a Single RGB Image](https://mlsanity.com/paper/1802.10367/) (2018)
- [Occlusion Resistant Object Rotation Regression from Point Cloud Segments](https://mlsanity.com/paper/1808.05498/) (2018)
- [Loss it right: Euclidean and Riemannian Metrics in Learning-based Visual Odometry](https://mlsanity.com/paper/2401.05396/) (2023)
- [Leveraging Anthropometric Measurements to Improve Human Mesh Estimation and Ensure Consistent Body Shapes](https://mlsanity.com/paper/2409.17671/) (2024)

## Built on by (most cited first, 1 of 1)

- [DuetHOI: Language-Guided Bimanual Hand--Object Motion Generation with Articulation Planning and Contact Refinement](https://mlsanity.com/paper/2603.08390/) (2026)

## Abstract (arXiv)

In sports analytics, accurately capturing both the 3D locations and rotations of body joints is essential for understanding an athlete's biomechanics. While Human Mesh Recovery (HMR) models can estimate joint rotations, they often exhibit lower accuracy in joint localization compared to 3D Human Pose Estimation (HPE) models. Recent work addressed this limitation by combining a 3D HPE model with inverse kinematics (IK) to estimate both joint locations and rotations. However, IK is computationally expensive. To overcome this, we propose a novel 2D-to-3D uplifting model that directly estimates 3D human poses, including joint rotations, in a single forward pass. We investigate multiple rotation representations, loss functions, and training strategies - both with and without access to ground truth rotations. Our models achieve state-of-the-art accuracy in rotation estimation, are 150 times faster than the IK-based approach, and surpass HMR models in joint localization precision.