# A hybrid classification-regression approach for 3D hand pose estimation using graph convolutional networks

arXiv: https://arxiv.org/abs/2105.10902 · HTML: https://mlsanity.com/paper/2105.10902/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 46 references that are in the map.
- First posted on arXiv 23 May 2021.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A simple yet effective baseline for 3d human pose estimation](https://mlsanity.com/paper/1705.03098/) (2017)
- [FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Images](https://mlsanity.com/paper/1909.04349/) (2019)
- [Learning to Estimate 3D Hand Pose from Single RGB Images](https://mlsanity.com/paper/1705.01389/) (2017)
- [Hand Keypoint Detection in Single Images using Multiview Bootstrapping](https://mlsanity.com/paper/1704.07809/) (2017)
- [Semantic Graph Convolutional Networks for 3D Human Pose Regression](https://mlsanity.com/paper/1904.03345/) (2019)
- [3D Hand Shape and Pose from Images in the Wild](https://mlsanity.com/paper/1902.03451/) (2019)
- [3D Hand Shape and Pose Estimation from a Single RGB Image](https://mlsanity.com/paper/1903.00812/) (2019)
- [GANerated Hands for Real-time 3D Hand Tracking from Monocular RGB](https://mlsanity.com/paper/1712.01057/) (2017)
- [End-to-end Hand Mesh Recovery from a Monocular RGB Image](https://mlsanity.com/paper/1902.09305/) (2019)
- [Pushing the Envelope for RGB-based Dense 3D Hand Pose Estimation via Neural Rendering](https://mlsanity.com/paper/1904.04196/) (2019)
- [HOPE-Net: A Graph-based Model for Hand-Object Pose Estimation](https://mlsanity.com/paper/2004.00060/) (2020)
- [Hand Pose Estimation via Latent 2.5D Heatmap Regression](https://mlsanity.com/paper/1804.09534/) (2018)
- [Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data](https://mlsanity.com/paper/2003.09572/) (2020)
- [V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Map](https://mlsanity.com/paper/1711.07399/) (2017)
- [Cross-modal Deep Variational Hand Pose Estimation](https://mlsanity.com/paper/1803.11404/) (2018)
- [Linkage Based Face Clustering via Graph Convolution Network](https://mlsanity.com/paper/1903.11306/) (2019)
- [Disentangling Latent Hands for Image Synthesis and Pose Estimation](https://mlsanity.com/paper/1812.01002/) (2018)
- [Using a single RGB frame for real time 3D hand pose estimation in the wild](https://mlsanity.com/paper/1712.03866/) (2017)
- [3D Hand Pose Tracking and Estimation Using Stereo Matching](https://mlsanity.com/paper/1610.07214/) (2016)
- [Knowledge as Priors: Cross-Modal Knowledge Generalization for Datasets without Superior Knowledge](https://mlsanity.com/paper/2004.00176/) (2020)
- [Point-to-Pose Voting based Hand Pose Estimation using Residual Permutation Equivariant Layer](https://mlsanity.com/paper/1812.02050/) (2018)
- [Large-scale Multiview 3D Hand Pose Dataset](https://mlsanity.com/paper/1707.03742/) (2017)
- [DeepPrior++: Improving Fast and Accurate 3D Hand Pose Estimation](https://mlsanity.com/paper/1708.08325/) (2017)
- [Crossing Nets: Combining GANs and VAEs with a Shared Latent Space for Hand Pose Estimation](https://mlsanity.com/paper/1702.03431/) (2017)

## Built on by (most cited first, 2 of 2)

- [ProtoSeg: A Prototype-Based Point Cloud Instance Segmentation Method](https://mlsanity.com/paper/2410.02352/) (2024)
- [End-to-End Steering for Autonomous Vehicles via Conditional Imitation Co-Learning](https://mlsanity.com/paper/2411.16131/) (2024)

## Abstract (arXiv)

Hand pose estimation is a crucial part of a wide range of augmented reality and human-computer interaction applications. Predicting the 3D hand pose from a single RGB image is challenging due to occlusion and depth ambiguities. GCN-based (Graph Convolutional Networks) methods exploit the structural relationship similarity between graphs and hand joints to model kinematic dependencies between joints. These techniques use predefined or globally learned joint relationships, which may fail to capture pose-dependent constraints. To address this problem, we propose a two-stage GCN-based framework that learns per-pose relationship constraints. Specifically, the first phase quantizes the 2D/3D space to classify the joints into 2D/3D blocks based on their locality. This spatial dependency information guides this phase to estimate reliable 2D and 3D poses. The second stage further improves the 3D estimation through a GCN-based module that uses an adaptative nearest neighbor algorithm to determine joint relationships. Extensive experiments show that our multi-stage GCN approach yields an efficient model that produces accurate 2D/3D hand poses and outperforms the state-of-the-art on two public datasets.