# Skeleton-aware multi-scale heatmap regression for 2D hand pose estimation

arXiv: https://arxiv.org/abs/2105.10904 · HTML: https://mlsanity.com/paper/2105.10904/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 20 references that are in the map.
- First posted on arXiv 23 May 2021.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Learning to Estimate 3D Hand Pose from Single RGB Images](https://mlsanity.com/paper/1705.01389/) (2017)
- [Hand Keypoint Detection in Single Images using Multiview Bootstrapping](https://mlsanity.com/paper/1704.07809/) (2017)
- [GANerated Hands for Real-time 3D Hand Tracking from Monocular RGB](https://mlsanity.com/paper/1712.01057/) (2017)
- [Towards Accurate Multi-person Pose Estimation in the Wild](https://mlsanity.com/paper/1701.01779/) (2017)
- [Human Pose Estimation with Iterative Error Feedback](https://mlsanity.com/paper/1507.06550/) (2015)
- [Hand Pose Estimation via Latent 2.5D Heatmap Regression](https://mlsanity.com/paper/1804.09534/) (2018)
- [Cross-modal Deep Variational Hand Pose Estimation](https://mlsanity.com/paper/1803.11404/) (2018)
- [Human pose estimation via Convolutional Part Heatmap Regression](https://mlsanity.com/paper/1609.01743/) (2016)
- [Point-to-Pose Voting based Hand Pose Estimation using Residual Permutation Equivariant Layer](https://mlsanity.com/paper/1812.02050/) (2018)
- [Large-scale Multiview 3D Hand Pose Dataset](https://mlsanity.com/paper/1707.03742/) (2017)
- [Crossing Nets: Combining GANs and VAEs with a Shared Latent Space for Hand Pose Estimation](https://mlsanity.com/paper/1702.03431/) (2017)
- [Adaptive Graphical Model Network for 2D Handpose Estimation](https://mlsanity.com/paper/1909.08205/) (2019)

## Built on by (most cited first, 1 of 1)

- [FaceTouch: Detecting hand-to-face touch with supervised contrastive learning to assist in tracing infectious disease](https://mlsanity.com/paper/2308.12840/) (2023)

## Abstract (arXiv)

Existing RGB-based 2D hand pose estimation methods learn the joint locations from a single resolution, which is not suitable for different hand sizes. To tackle this problem, we propose a new deep learning-based framework that consists of two main modules. The former presents a segmentation-based approach to detect the hand skeleton and localize the hand bounding box. The second module regresses the 2D joint locations through a multi-scale heatmap regression approach that exploits the predicted hand skeleton as a constraint to guide the model. Furthermore, we construct a new dataset that is suitable for both hand detection and pose estimation. We qualitatively and quantitatively validate our method on two datasets. Results demonstrate that the proposed method outperforms state-of-the-art and can recover the pose even in cluttered images and complex poses.