# Depth-Based 3D Hand Pose Estimation: From Current Achievements to Future Goals

arXiv: https://arxiv.org/abs/1712.03917 · HTML: https://mlsanity.com/paper/1712.03917/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Dec 2017.

## Built on by (most cited first, 21 of 21)

- [H2O: Two Hands Manipulating Objects for First Person Interaction Recognition](https://mlsanity.com/paper/2104.11181/) (2021)
- [Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Time](https://mlsanity.com/paper/2106.05266/) (2021)
- [Discrete Cosine Transform Network for Guided Depth Map Super-Resolution](https://mlsanity.com/paper/2104.06977/) (2021)
- [Real-time Pose and Shape Reconstruction of Two Interacting Hands With a Single Depth Camera](https://mlsanity.com/paper/2106.08059/) (2021)
- [Collaborative Learning for Hand and Object Reconstruction with Attention-guided Graph Convolution](https://mlsanity.com/paper/2204.13062/) (2022)
- [I2UV-HandNet: Image-to-UV Prediction Network for Accurate and High-fidelity 3D Hand Mesh Modeling](https://mlsanity.com/paper/2102.03725/) (2021)
- [gSDF: Geometry-Driven Signed Distance Functions for 3D Hand-Object Reconstruction](https://mlsanity.com/paper/2304.11970/) (2023)
- [Learning to Imitate Object Interactions from Internet Videos](https://mlsanity.com/paper/2211.13225/) (2022)
- [Hand Image Understanding via Deep Multi-Task Learning](https://mlsanity.com/paper/2107.11646/) (2021)
- [Consistent 3D Hand Reconstruction in Video via self-supervised Learning](https://mlsanity.com/paper/2201.09548/) (2022)
- [Spherical Space Feature Decomposition for Guided Depth Map Super-Resolution](https://mlsanity.com/paper/2303.08942/) (2023)
- [RGB2Hands: Real-Time Tracking of 3D Hand Interactions from Monocular RGB Video](https://mlsanity.com/paper/2106.11725/) (2021)
- [Efficient Annotation and Learning for 3D Hand Pose Estimation: A Survey](https://mlsanity.com/paper/2206.02257/) (2022)
- [TempCLR: Reconstructing Hands via Time-Coherent Contrastive Learning](https://mlsanity.com/paper/2209.00489/) (2022)
- [HandTailor: Towards High-Precision Monocular 3D Hand Recovery](https://mlsanity.com/paper/2102.09244/) (2021)
- [A2J-Transformer: Anchor-to-Joint Transformer Network for 3D Interacting Hand Pose Estimation from a Single RGB Image](https://mlsanity.com/paper/2304.03635/) (2023)
- [HandVoxNet++: 3D Hand Shape and Pose Estimation using Voxel-Based Neural Networks](https://mlsanity.com/paper/2107.01205/) (2021)
- [TriHorn-Net: A Model for Accurate Depth-Based 3D Hand Pose Estimation](https://mlsanity.com/paper/2206.07117/) (2022)
- [End-to-end Weakly-supervised Single-stage Multiple 3D Hand Mesh Reconstruction from a Single RGB Image](https://mlsanity.com/paper/2204.08154/) (2022)
- [Pushing the Envelope for Depth-Based Semi-Supervised 3D Hand Pose Estimation with Consistency Training](https://mlsanity.com/paper/2303.15147/) (2023)
- [Szloca: towards a framework for full 3D tracking through a single camera in context of interactive arts](https://mlsanity.com/paper/2206.12958/) (2022)

## Abstract (arXiv)

In this paper, we strive to answer two questions: What is the current state of 3D hand pose estimation from depth images? And, what are the next challenges that need to be tackled? Following the successful Hands In the Million Challenge (HIM2017), we investigate the top 10 state-of-the-art methods on three tasks: single frame 3D pose estimation, 3D hand tracking, and hand pose estimation during object interaction. We analyze the performance of different CNN structures with regard to hand shape, joint visibility, view point and articulation distributions. Our findings include: (1) isolated 3D hand pose estimation achieves low mean errors (10 mm) in the view point range of [70, 120] degrees, but it is far from being solved for extreme view points; (2) 3D volumetric representations outperform 2D CNNs, better capturing the spatial structure of the depth data; (3) Discriminative methods still generalize poorly to unseen hand shapes; (4) While joint occlusions pose a challenge for most methods, explicit modeling of structure constraints can significantly narrow the gap between errors on visible and occluded joints.