# Two-hand Global 3D Pose Estimation Using Monocular RGB

arXiv: https://arxiv.org/abs/2006.01320 · HTML: https://mlsanity.com/paper/2006.01320/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jun 2020.

## Built on by (most cited first, 27 of 27)

- [IMos: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactions](https://mlsanity.com/paper/2212.07555/) (2022)
- [Fine-Grained Egocentric Hand-Object Segmentation: Dataset, Model, and Applications](https://mlsanity.com/paper/2208.03826/) (2022)
- [Learning to Disambiguate Strongly Interacting Hands via Probabilistic Per-pixel Part Segmentation](https://mlsanity.com/paper/2107.00434/) (2021)
- [3D Interacting Hand Pose Estimation by Hand De-occlusion and Removal](https://mlsanity.com/paper/2207.11061/) (2022)
- [Reconstructing Hand-Held Objects from Monocular Video](https://mlsanity.com/paper/2211.16835/) (2022)
- [A Dataset of Relighted 3D Interacting Hands](https://mlsanity.com/paper/2310.17768/) (2023)
- [Lightweight Multi-person Total Motion Capture Using Sparse Multi-view Cameras](https://mlsanity.com/paper/2108.10378/) (2021)
- [Reconstructing Interacting Hands with Interaction Prior from Monocular Images](https://mlsanity.com/paper/2308.14082/) (2023)
- [RenderIH: A Large-scale Synthetic Dataset for 3D Interacting Hand Pose Estimation](https://mlsanity.com/paper/2309.09301/) (2023)
- [A2J-Transformer: Anchor-to-Joint Transformer Network for 3D Interacting Hand Pose Estimation from a Single RGB Image](https://mlsanity.com/paper/2304.03635/) (2023)
- [HandDiffuse: Generative Controllers for Two-Hand Interactions via Diffusion Models](https://mlsanity.com/paper/2312.04867/) (2023)
- [Spectral Graphormer: Spectral Graph-based Transformer for Egocentric Two-Hand Reconstruction using Multi-View Color Images](https://mlsanity.com/paper/2308.11015/) (2023)
- [AttentionHand: Text-driven Controllable Hand Image Generation for 3D Hand Reconstruction in the Wild](https://mlsanity.com/paper/2407.18034/) (2024)
- [Egocentric Hand-object Interaction Detection and Application](https://mlsanity.com/paper/2109.14734/) (2021)
- [Understanding Egocentric Hand-Object Interactions from Hand Pose Estimation](https://mlsanity.com/paper/2109.14657/) (2021)
- [HandFlow: Quantifying View-Dependent 3D Ambiguity in Two-Hand Reconstruction with Normalizing Flow](https://mlsanity.com/paper/2210.01692/) (2022)
- [Ego2HandsPose: A Dataset for Egocentric Two-hand 3D Global Pose Estimation](https://mlsanity.com/paper/2206.04927/) (2022)
- [Image-free Domain Generalization via CLIP for 3D Hand Pose Estimation](https://mlsanity.com/paper/2210.16788/) (2022)
- [ThermoHands: A Benchmark for 3D Hand Pose Estimation from Egocentric Thermal Images](https://mlsanity.com/paper/2403.09871/) (2024)
- [Benchmarking 2D Egocentric Hand Pose Datasets](https://mlsanity.com/paper/2409.07337/) (2024)
- [Semi-supervised Hand Appearance Recovery via Structure Disentanglement and Dual Adversarial Discrimination](https://mlsanity.com/paper/2303.06380/) (2023)
- [V-Hands: Touchscreen-based Hand Tracking for Remote Whiteboard Interaction](https://mlsanity.com/paper/2409.13347/) (2024)
- [VM-BHINet:Vision Mamba Bimanual Hand Interaction Network for 3D Interacting Hand Mesh Recovery From a Single RGB Image](https://mlsanity.com/paper/2504.14618/) (2025)
- [AVI-HT: Adaptive Vision-IMU Fusion for 3D Hand Tracking](https://mlsanity.com/paper/2605.21714/) (2026)
- [Exploiting Spatial-Temporal Context for Interacting Hand Reconstruction on Monocular RGB Video](https://mlsanity.com/paper/2308.04074/) (2023)
- [Parallel mesh reconstruction streams for pose estimation of interacting hands](https://mlsanity.com/paper/2104.12123/) (2021)
- [Egocentric Hand-object Interaction Detection](https://mlsanity.com/paper/2211.09067/) (2022)

## Abstract (arXiv)

We tackle the challenging task of estimating global 3D joint locations for both hands via only monocular RGB input images. We propose a novel multi-stage convolutional neural network based pipeline that accurately segments and locates the hands despite occlusion between two hands and complex background noise and estimates the 2D and 3D canonical joint locations without any depth information. Global joint locations with respect to the camera origin are computed using the hand pose estimations and the actual length of the key bone with a novel projection algorithm. To train the CNNs for this new task, we introduce a large-scale synthetic 3D hand pose dataset. We demonstrate that our system outperforms previous works on 3D canonical hand pose estimation benchmark datasets with RGB-only information. Additionally, we present the first work that achieves accurate global 3D hand tracking on both hands using RGB-only inputs and provide extensive quantitative and qualitative evaluation.