# Depth-based hand pose estimation: methods, data, and challenges

arXiv: https://arxiv.org/abs/1504.06378 · HTML: https://mlsanity.com/paper/1504.06378/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Apr 2015.

## Built on by (most cited first, 16 of 16)

- [Embodied Hands: Modeling and Capturing Hands and Bodies Together](https://mlsanity.com/paper/2201.02610/) (2022)
- [Real-time Pose and Shape Reconstruction of Two Interacting Hands With a Single Depth Camera](https://mlsanity.com/paper/2106.08059/) (2021)
- [AlignSDF: Pose-Aligned Signed Distance Fields for Hand-Object Reconstruction](https://mlsanity.com/paper/2207.12909/) (2022)
- [emg2pose: A Large and Diverse Benchmark for Surface Electromyographic Hand Pose Estimation](https://mlsanity.com/paper/2412.02725/) (2024)
- [HUMBI: A Large Multiview Dataset of Human Body Expressions and Benchmark Challenge](https://mlsanity.com/paper/2110.00119/) (2021)
- [Efficient Annotation and Learning for 3D Hand Pose Estimation: A Survey](https://mlsanity.com/paper/2206.02257/) (2022)
- [TempCLR: Reconstructing Hands via Time-Coherent Contrastive Learning](https://mlsanity.com/paper/2209.00489/) (2022)
- [EvHandPose: Event-based 3D Hand Pose Estimation with Sparse Supervision](https://mlsanity.com/paper/2303.02862/) (2023)
- [HandTailor: Towards High-Precision Monocular 3D Hand Recovery](https://mlsanity.com/paper/2102.09244/) (2021)
- [HandVoxNet++: 3D Hand Shape and Pose Estimation using Voxel-Based Neural Networks](https://mlsanity.com/paper/2107.01205/) (2021)
- [Survey on Hand Gesture Recognition from Visual Input](https://mlsanity.com/paper/2501.11992/) (2025)
- [Multi-View Video-Based 3D Hand Pose Estimation](https://mlsanity.com/paper/2109.11747/) (2021)
- [Learning Sequential Contexts using Transformer for 3D Hand Pose Estimation](https://mlsanity.com/paper/2206.00171/) (2022)
- [Pushing the Envelope for Depth-Based Semi-Supervised 3D Hand Pose Estimation with Consistency Training](https://mlsanity.com/paper/2303.15147/) (2023)
- [Real-time SLAM Pipeline in Dynamics Environment](https://mlsanity.com/paper/2303.02272/) (2023)
- [VEPHand: View-Efficient Photometric Hand Performance Capture at Scale](https://mlsanity.com/paper/2606.15966/) (2026)

## Abstract (arXiv)

Hand pose estimation has matured rapidly in recent years. The introduction of commodity depth sensors and a multitude of practical applications have spurred new advances. We provide an extensive analysis of the state-of-the-art, focusing on hand pose estimation from a single depth frame. To do so, we have implemented a considerable number of systems, and will release all software and evaluation code. We summarize important conclusions here: (1) Pose estimation appears roughly solved for scenes with isolated hands. However, methods still struggle to analyze cluttered scenes where hands may be interacting with nearby objects and surfaces. To spur further progress we introduce a challenging new dataset with diverse, cluttered scenes. (2) Many methods evaluate themselves with disparate criteria, making comparisons difficult. We define a consistent evaluation criteria, rigorously motivated by human experiments. (3) We introduce a simple nearest-neighbor baseline that outperforms most existing systems. This implies that most systems do not generalize beyond their training sets. This also reinforces the under-appreciated point that training data is as important as the model itself. We conclude with directions for future progress.