# 3D Hand Pose Tracking and Estimation Using Stereo Matching

arXiv: https://arxiv.org/abs/1610.07214 · HTML: https://mlsanity.com/paper/1610.07214/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Oct 2016.

## Built on by (most cited first, 30 of 35)

- [FrankMocap: A Monocular 3D Whole-Body Pose Estimation System via Regression and Integration](https://mlsanity.com/paper/2108.06428/) (2021)
- [HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videos](https://mlsanity.com/paper/2411.19167/) (2024)
- [MobRecon: Mobile-Friendly Hand Mesh Reconstruction from Monocular Image](https://mlsanity.com/paper/2112.02753/) (2021)
- [I2UV-HandNet: Image-to-UV Prediction Network for Accurate and High-fidelity 3D Hand Mesh Modeling](https://mlsanity.com/paper/2102.03725/) (2021)
- [Introducing HOT3D: An Egocentric Dataset for 3D Hand and Object Tracking](https://mlsanity.com/paper/2406.09598/) (2024)
- [A Dataset of Relighted 3D Interacting Hands](https://mlsanity.com/paper/2310.17768/) (2023)
- [Contrastive Representation Learning for Hand Shape Estimation](https://mlsanity.com/paper/2106.04324/) (2021)
- [DART: Articulated Hand Model with Diverse Accessories and Rich Textures](https://mlsanity.com/paper/2210.07650/) (2022)
- [EvHandPose: Event-based 3D Hand Pose Estimation with Sparse Supervision](https://mlsanity.com/paper/2303.02862/) (2023)
- [RenderIH: A Large-scale Synthetic Dataset for 3D Interacting Hand Pose Estimation](https://mlsanity.com/paper/2309.09301/) (2023)
- [Dense Hand-Object(HO) GraspNet with Full Grasping Taxonomy and Dynamics](https://mlsanity.com/paper/2409.04033/) (2024)
- [HandDiffuse: Generative Controllers for Two-Hand Interactions via Diffusion Models](https://mlsanity.com/paper/2312.04867/) (2023)
- [Survey on Hand Gesture Recognition from Visual Input](https://mlsanity.com/paper/2501.11992/) (2025)
- [Multi-View Video-Based 3D Hand Pose Estimation](https://mlsanity.com/paper/2109.11747/) (2021)
- [Local and Global Point Cloud Reconstruction for 3D Hand Pose Estimation](https://mlsanity.com/paper/2112.06389/) (2021)
- [RealisticHands: A Hybrid Model for 3D Hand Reconstruction](https://mlsanity.com/paper/2108.13995/) (2021)
- [3D Hand Pose and Shape Estimation from RGB Images for Keypoint-Based Hand Gesture Recognition](https://mlsanity.com/paper/2109.13879/) (2021)
- [CLIP-Hand3D: Exploiting 3D Hand Pose Estimation via Context-Aware Prompting](https://mlsanity.com/paper/2309.16140/) (2023)
- [Weakly-Supervised 3D Hand Reconstruction with Knowledge Prior and Uncertainty Guidance](https://mlsanity.com/paper/2407.12307/) (2024)
- [Dynamic Iterative Refinement for Efficient 3D Hand Pose Estimation](https://mlsanity.com/paper/2111.06500/) (2021)
- [PALM: A Dataset and Baseline for Learning Multi-subject Hand Prior](https://mlsanity.com/paper/2511.05403/) (2025)
- [Temporally Guided Articulated Hand Pose Tracking in Surgical Videos](https://mlsanity.com/paper/2101.04281/) (2021)
- [A hybrid classification-regression approach for 3D hand pose estimation using graph convolutional networks](https://mlsanity.com/paper/2105.10902/) (2021)
- [Semi-supervised Hand Appearance Recovery via Structure Disentanglement and Dual Adversarial Discrimination](https://mlsanity.com/paper/2303.06380/) (2023)
- [Denoising Diffusion for 3D Hand Pose Estimation from Images](https://mlsanity.com/paper/2308.09523/) (2023)
- [HandOcc: NeRF-based Hand Rendering with Occupancy Networks](https://mlsanity.com/paper/2505.02079/) (2025)
- [Fast Monocular Hand Pose Estimation on Embedded Systems](https://mlsanity.com/paper/2102.07067/) (2021)
- [Learning Sequential Contexts using Transformer for 3D Hand Pose Estimation](https://mlsanity.com/paper/2206.00171/) (2022)
- [End-to-End 3D Hand Pose Estimation from Stereo Cameras](https://mlsanity.com/paper/2206.01384/) (2022)
- [Self-supervised Latent Space Optimization with Nebula Variational Coding](https://mlsanity.com/paper/2506.01414/) (2025)

## Abstract (arXiv)

3D hand pose tracking/estimation will be very important in the next generation of human-computer interaction. Most of the currently available algorithms rely on low-cost active depth sensors. However, these sensors can be easily interfered by other active sources and require relatively high power consumption. As a result, they are currently not suitable for outdoor environments and mobile devices. This paper aims at tracking/estimating hand poses using passive stereo which avoids these limitations. A benchmark with 18,000 stereo image pairs and 18,000 depth images captured from different scenarios and the ground-truth 3D positions of palm and finger joints (obtained from the manual label) is thus proposed. This paper demonstrates that the performance of the state-of-the art tracking/estimation algorithms can be maintained with most stereo matching algorithms on the proposed benchmark, as long as the hand segmentation is correct. As a result, a novel stereo-based hand segmentation algorithm specially designed for hand tracking/estimation is proposed. The quantitative evaluation demonstrates that the proposed algorithm is suitable for the state-of-the-art hand pose tracking/estimation algorithms and the tracking quality is comparable to the use of active depth sensors under different challenging scenarios.