# MM-Hand: 3D-Aware Multi-Modal Guided Hand Generative Network for 3D Hand Pose Synthesis

arXiv: https://arxiv.org/abs/2010.01158 · HTML: https://mlsanity.com/paper/2010.01158/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Oct 2020.

## Built on by (most cited first, 5 of 5)

- [SignDiff: Diffusion Model for American Sign Language Production](https://mlsanity.com/paper/2308.16082/) (2023)
- [Hand-Object Interaction Image Generation](https://mlsanity.com/paper/2211.15663/) (2022)
- [CLIP-Hand3D: Exploiting 3D Hand Pose Estimation via Context-Aware Prompting](https://mlsanity.com/paper/2309.16140/) (2023)
- [Black-Box Diagnosis and Calibration on GAN Intra-Mode Collapse: A Pilot Study](https://mlsanity.com/paper/2107.12202/) (2021)
- [Single Image 3D Object Estimation with Primitive Graph Networks](https://mlsanity.com/paper/2109.04153/) (2021)

## Abstract (arXiv)

Estimating the 3D hand pose from a monocular RGB image is important but challenging. A solution is training on large-scale RGB hand images with accurate 3D hand keypoint annotations. However, it is too expensive in practice. Instead, we have developed a learning-based approach to synthesize realistic, diverse, and 3D pose-preserving hand images under the guidance of 3D pose information. We propose a 3D-aware multi-modal guided hand generative network (MM-Hand), together with a novel geometry-based curriculum learning strategy. Our extensive experimental results demonstrate that the 3D-annotated images generated by MM-Hand qualitatively and quantitatively outperform existing options. Moreover, the augmented data can consistently improve the quantitative performance of the state-of-the-art 3D hand pose estimators on two benchmark datasets. The code will be available at https://github.com/ScottHoang/mm-hand.