# HOPE-Net: A Graph-based Model for Hand-Object Pose Estimation

arXiv: https://arxiv.org/abs/2004.00060 · HTML: https://mlsanity.com/paper/2004.00060/

## Key facts

- Cited by 112 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Mar 2020.

## Built on by (most cited first, 30 of 112)

- [DexYCB: A Benchmark for Capturing Hand Grasping of Objects](https://mlsanity.com/paper/2104.04631/) (2021)
- [ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation](https://mlsanity.com/paper/2204.13662/) (2022)
- [Hand-Object Contact Consistency Reasoning for Human Grasps Generation](https://mlsanity.com/paper/2104.03304/) (2021)
- [Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Time](https://mlsanity.com/paper/2106.05266/) (2021)
- [ContactOpt: Optimizing Contact to Improve Grasps](https://mlsanity.com/paper/2104.07267/) (2021)
- [OakInk: A Large-scale Knowledge Repository for Understanding Hand-Object Interaction](https://mlsanity.com/paper/2203.15709/) (2022)
- [D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactions](https://mlsanity.com/paper/2112.03028/) (2021)
- [OAKINK2: A Dataset of Bimanual Hands-Object Manipulation in Complex Task Completion](https://mlsanity.com/paper/2403.19417/) (2024)
- [ArtiBoost: Boosting Articulated 3D Hand-Object Pose Estimation via Online Exploration and Synthesis](https://mlsanity.com/paper/2109.05488/) (2021)
- [Camera-Space Hand Mesh Recovery via Semantic Aggregation and Adaptive 2D-1D Registration](https://mlsanity.com/paper/2103.02845/) (2021)
- [AlignSDF: Pose-Aligned Signed Distance Fields for Hand-Object Reconstruction](https://mlsanity.com/paper/2207.12909/) (2022)
- [Towards unconstrained joint hand-object reconstruction from RGB videos](https://mlsanity.com/paper/2108.07044/) (2021)
- [ParaHome: Parameterizing Everyday Home Activities Towards 3D Generative Modeling of Human-Object Interactions](https://mlsanity.com/paper/2401.10232/) (2024)
- [CAMS: CAnonicalized Manipulation Spaces for Category-Level Functional Hand-Object Manipulation Synthesis](https://mlsanity.com/paper/2303.15469/) (2023)
- [Collaborative Learning for Hand and Object Reconstruction with Attention-guided Graph Convolution](https://mlsanity.com/paper/2204.13062/) (2022)
- [ContactGen: Generative Contact Modeling for Grasp Generation](https://mlsanity.com/paper/2310.03740/) (2023)
- [A Skeleton-Driven Neural Occupancy Representation for Articulated Hands](https://mlsanity.com/paper/2109.11399/) (2021)
- [AffordPose: A Large-scale Dataset of Hand-Object Interactions with Affordance-driven Hand Pose](https://mlsanity.com/paper/2309.08942/) (2023)
- [WarpedGANSpace: Finding non-linear RBF paths in GAN latent space](https://mlsanity.com/paper/2109.13357/) (2021)
- [Towards Accurate Alignment in Real-time 3D Hand-Mesh Reconstruction](https://mlsanity.com/paper/2109.01723/) (2021)
- [Learning to Disambiguate Strongly Interacting Hands via Probabilistic Per-pixel Part Segmentation](https://mlsanity.com/paper/2107.00434/) (2021)
- [Deep Learning-Based Object Pose Estimation: A Comprehensive Survey](https://mlsanity.com/paper/2405.07801/) (2024)
- [HOIDiffusion: Generating Realistic 3D Hand-Object Interaction Data](https://mlsanity.com/paper/2403.12011/) (2024)
- [Realistic and Efficient Face Swapping: A Unified Approach with Diffusion Models](https://mlsanity.com/paper/2409.07269/) (2024)
- [DAD-3DHeads: A Large-scale Dense, Accurate and Diverse Dataset for 3D Head Alignment from a Single Image](https://mlsanity.com/paper/2204.03688/) (2022)
- [HARP: Personalized Hand Reconstruction from a Monocular RGB Video](https://mlsanity.com/paper/2212.09530/) (2022)
- [Interaction Replica: Tracking Human-Object Interaction and Scene Changes From Human Motion](https://mlsanity.com/paper/2205.02830/) (2022)
- [Im2Hands: Learning Attentive Implicit Representation of Interacting Two-Hand Shapes](https://mlsanity.com/paper/2302.14348/) (2023)
- [ZoomNAS: Searching for Whole-body Human Pose Estimation in the Wild](https://mlsanity.com/paper/2208.11547/) (2022)
- [HMP: Hand Motion Priors for Pose and Shape Estimation from Video](https://mlsanity.com/paper/2312.16737/) (2023)

## Abstract (arXiv)

Hand-object pose estimation (HOPE) aims to jointly detect the poses of both a hand and of a held object. In this paper, we propose a lightweight model called HOPE-Net which jointly estimates hand and object pose in 2D and 3D in real-time. Our network uses a cascade of two adaptive graph convolutional neural networks, one to estimate 2D coordinates of the hand joints and object corners, followed by another to convert 2D coordinates to 3D. Our experiments show that through end-to-end training of the full network, we achieve better accuracy for both the 2D and 3D coordinate estimation problems. The proposed 2D to 3D graph convolution-based model could be applied to other 3D landmark detection problems, where it is possible to first predict the 2D keypoints and then transform them to 3D.