# DeepHPS: End-to-end Estimation of 3D Hand Pose and Shape by Learning from Synthetic Depth

arXiv: https://arxiv.org/abs/1808.09208 · HTML: https://mlsanity.com/paper/1808.09208/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Aug 2018.

## Built on by (most cited first, 6 of 6)

- [Towards Accurate Alignment in Real-time 3D Hand-Mesh Reconstruction](https://mlsanity.com/paper/2109.01723/) (2021)
- [Learning Feature Aggregation for Deep 3D Morphable Models](https://mlsanity.com/paper/2105.02173/) (2021)
- [HandVoxNet++: 3D Hand Shape and Pose Estimation using Voxel-Based Neural Networks](https://mlsanity.com/paper/2107.01205/) (2021)
- [Survey on Hand Gesture Recognition from Visual Input](https://mlsanity.com/paper/2501.11992/) (2025)
- [ShapeGraFormer: GraFormer-Based Network for Hand-Object Reconstruction from a Single Depth Map](https://mlsanity.com/paper/2310.11811/) (2023)
- [BIDCD -- Bosch Industrial Depth Completion Dataset](https://mlsanity.com/paper/2108.04706/) (2021)

## Abstract (arXiv)

Articulated hand pose and shape estimation is an important problem for vision-based applications such as augmented reality and animation. In contrast to the existing methods which optimize only for joint positions, we propose a fully supervised deep network which learns to jointly estimate a full 3D hand mesh representation and pose from a single depth image. To this end, a CNN architecture is employed to estimate parametric representations i.e. hand pose, bone scales and complex shape parameters. Then, a novel hand pose and shape layer, embedded inside our deep framework, produces 3D joint positions and hand mesh. Lack of sufficient training data with varying hand shapes limits the generalized performance of learning based methods. Also, manually annotating real data is suboptimal. Therefore, we present SynHand5M: a million-scale synthetic dataset with accurate joint annotations, segmentation masks and mesh files of depth maps. Among model based learning (hybrid) methods, we show improved results on our dataset and two of the public benchmarks i.e. NYU and ICVL. Also, by employing a joint training strategy with real and synthetic data, we recover 3D hand mesh and pose from real images in 3.7ms.