# Generalized Feedback Loop for Joint Hand-Object Pose Estimation

arXiv: https://arxiv.org/abs/1903.10883 · HTML: https://mlsanity.com/paper/1903.10883/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Mar 2019.

## Built on by (most cited first, 22 of 22)

- [PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop](https://mlsanity.com/paper/2103.16507/) (2021)
- [Hand-Object Contact Consistency Reasoning for Human Grasps Generation](https://mlsanity.com/paper/2104.03304/) (2021)
- [PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Images](https://mlsanity.com/paper/2207.06400/) (2022)
- [Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Time](https://mlsanity.com/paper/2106.05266/) (2021)
- [HOIDiffusion: Generating Realistic 3D Hand-Object Interaction Data](https://mlsanity.com/paper/2403.12011/) (2024)
- [Single-view robot pose and joint angle estimation via render & compare](https://mlsanity.com/paper/2104.09359/) (2021)
- [Interaction Replica: Tracking Human-Object Interaction and Scene Changes From Human Motion](https://mlsanity.com/paper/2205.02830/) (2022)
- [Hand-Object Interaction Image Generation](https://mlsanity.com/paper/2211.15663/) (2022)
- [Focal Length and Object Pose Estimation via Render and Compare](https://mlsanity.com/paper/2204.05145/) (2022)
- [Tracking and Reconstructing Hand Object Interactions from Point Cloud Sequences in the Wild](https://mlsanity.com/paper/2209.12009/) (2022)
- [ContactArt: Learning 3D Interaction Priors for Category-level Articulated Object and Hand Poses Estimation](https://mlsanity.com/paper/2305.01618/) (2023)
- [SHOWMe: Benchmarking Object-agnostic Hand-Object 3D Reconstruction](https://mlsanity.com/paper/2309.10748/) (2023)
- [TriHorn-Net: A Model for Accurate Depth-Based 3D Hand Pose Estimation](https://mlsanity.com/paper/2206.07117/) (2022)
- [Weakly-Supervised 3D Hand Reconstruction with Knowledge Prior and Uncertainty Guidance](https://mlsanity.com/paper/2407.12307/) (2024)
- [FocalPose++: Focal Length and Object Pose Estimation via Render and Compare](https://mlsanity.com/paper/2312.02985/) (2023)
- [HOSt3R: Keypoint-free Hand-Object 3D Reconstruction from RGB images](https://mlsanity.com/paper/2508.16465/) (2025)
- [Tracking Fast by Learning Slow: An Event-based Speed Adaptive Hand Tracker Leveraging Knowledge in RGB Domain](https://mlsanity.com/paper/2302.14430/) (2023)
- [ShapeGraFormer: GraFormer-Based Network for Hand-Object Reconstruction from a Single Depth Map](https://mlsanity.com/paper/2310.11811/) (2023)
- [HUP-3D: A 3D multi-view synthetic dataset for assisted-egocentric hand-ultrasound pose estimation](https://mlsanity.com/paper/2407.09215/) (2024)
- [Semi-supervised 3D Hand-Object Pose Estimation via Pose Dictionary Learning](https://mlsanity.com/paper/2107.07676/) (2021)
- [Pushing the Envelope for Depth-Based Semi-Supervised 3D Hand Pose Estimation with Consistency Training](https://mlsanity.com/paper/2303.15147/) (2023)
- [Serial-parallel Multi-Scale Feature Fusion for Anatomy-Oriented Hand Joint Detection](https://mlsanity.com/paper/2102.09757/) (2021)

## Abstract (arXiv)

We propose an approach to estimating the 3D pose of a hand, possibly handling an object, given a depth image. We show that we can correct the mistakes made by a Convolutional Neural Network trained to predict an estimate of the 3D pose by using a feedback loop. The components of this feedback loop are also Deep Networks, optimized using training data. This approach can be generalized to a hand interacting with an object. Therefore, we jointly estimate the 3D pose of the hand and the 3D pose of the object. Our approach performs en-par with state-of-the-art methods for 3D hand pose estimation, and outperforms state-of-the-art methods for joint hand-object pose estimation when using depth images only. Also, our approach is efficient as our implementation runs in real-time on a single GPU.