Generalized Feedback Loop for Joint Hand-Object Pose Estimation
Not on the map. Why?
Abstract (arXiv)
We propose an approach to estimating the 3D pose of a hand, possibly handling an object, given a depth image. We show that we can correct the mistakes made by a Convolutional Neural Network trained to predict an estimate of the 3D pose by using a feedback loop. The components of this feedback loop are also Deep Networks, optimized using training data. This approach can be generalized to a hand interacting with an object. Therefore, we jointly estimate the 3D pose of the hand and the 3D pose of the object. Our approach performs en-par with state-of-the-art methods for 3D hand pose estimation, and outperforms state-of-the-art methods for joint hand-object pose estimation when using depth images only. Also, our approach is efficient as our implementation runs in real-time on a single GPU.
Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Mar 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 6; 2022: 6; 2023: 6; 2024: 3; 2025: 1.
22 papers.
Sort:
By area:
- 2021PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loopcited by 250
- 2021Hand-Object Contact Consistency Reasoning for Human Grasps Generationcited by 160
- 2022PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Imagescited by 152
- 2021Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Timecited by 140
- 2024HOIDiffusion: Generating Realistic 3D Hand-Object Interaction Datacited by 37
- 2021Single-view robot pose and joint angle estimation via render & comparecited by 33
- 2022Interaction Replica: Tracking Human-Object Interaction and Scene Changes From Human Motioncited by 27
- 2022Hand-Object Interaction Image Generationcited by 19
- 2022Focal Length and Object Pose Estimation via Render and Comparecited by 17
- 2022Tracking and Reconstructing Hand Object Interactions from Point Cloud Sequences in the Wildcited by 17
- 2023ContactArt: Learning 3D Interaction Priors for Category-level Articulated Object and Hand Poses Estimationcited by 16
- 2023SHOWMe: Benchmarking Object-agnostic Hand-Object 3D Reconstructioncited by 16
10 more
- 2022TriHorn-Net: A Model for Accurate Depth-Based 3D Hand Pose Estimationcited by 7
- 2024Weakly-Supervised 3D Hand Reconstruction with Knowledge Prior and Uncertainty Guidancecited by 5
- 2023FocalPose++: Focal Length and Object Pose Estimation via Render and Comparecited by 3
- 2025HOSt3R: Keypoint-free Hand-Object 3D Reconstruction from RGB imagescited by 3
- 2023Tracking Fast by Learning Slow: An Event-based Speed Adaptive Hand Tracker Leveraging Knowledge in RGB Domaincited by 2
- 2023ShapeGraFormer: GraFormer-Based Network for Hand-Object Reconstruction from a Single Depth Mapcited by 2
- 2024HUP-3D: A 3D multi-view synthetic dataset for assisted-egocentric hand-ultrasound pose estimationcited by 2
- 2021Semi-supervised 3D Hand-Object Pose Estimation via Pose Dictionary Learningcited by 1
- 2023Pushing the Envelope for Depth-Based Semi-Supervised 3D Hand Pose Estimation with Consistency Trainingcited by 1
- 2021Serial-parallel Multi-Scale Feature Fusion for Anatomy-Oriented Hand Joint Detectioncited by 0
- 2025HOSt3R: Keypoint-free Hand-Object 3D Reconstruction from RGB images22 Aug 2025
- 2024Weakly-Supervised 3D Hand Reconstruction with Knowledge Prior and Uncertainty Guidance17 Jul 2024
- 2024HUP-3D: A 3D multi-view synthetic dataset for assisted-egocentric hand-ultrasound pose estimation12 Jul 2024
- 2024HOIDiffusion: Generating Realistic 3D Hand-Object Interaction Data18 Mar 2024
- 2023FocalPose++: Focal Length and Object Pose Estimation via Render and Compare15 Nov 2023
- 2023ShapeGraFormer: GraFormer-Based Network for Hand-Object Reconstruction from a Single Depth Map18 Oct 2023
- 2023SHOWMe: Benchmarking Object-agnostic Hand-Object 3D Reconstruction19 Sep 2023
- 2023ContactArt: Learning 3D Interaction Priors for Category-level Articulated Object and Hand Poses Estimation2 May 2023
- 2023Pushing the Envelope for Depth-Based Semi-Supervised 3D Hand Pose Estimation with Consistency Training27 Mar 2023
- 2023Tracking Fast by Learning Slow: An Event-based Speed Adaptive Hand Tracker Leveraging Knowledge in RGB Domain28 Feb 2023
- 2022Hand-Object Interaction Image Generation28 Nov 2022
- 2022Tracking and Reconstructing Hand Object Interactions from Point Cloud Sequences in the Wild24 Sep 2022
10 more
- 2022PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Images13 Jul 2022
- 2022TriHorn-Net: A Model for Accurate Depth-Based 3D Hand Pose Estimation14 Jun 2022
- 2022Interaction Replica: Tracking Human-Object Interaction and Scene Changes From Human Motion5 May 2022
- 2022Focal Length and Object Pose Estimation via Render and Compare11 Apr 2022
- 2021Semi-supervised 3D Hand-Object Pose Estimation via Pose Dictionary Learning16 Jul 2021
- 2021Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Time9 Jun 2021
- 2021Single-view robot pose and joint angle estimation via render & compare19 Apr 2021
- 2021Hand-Object Contact Consistency Reasoning for Human Grasps Generation7 Apr 2021
- 2021PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop30 Mar 2021
- 2021Serial-parallel Multi-Scale Feature Fusion for Anatomy-Oriented Hand Joint Detection19 Feb 2021
- 2021PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loopcited by 250
- 2021Hand-Object Contact Consistency Reasoning for Human Grasps Generationcited by 160
- 2022PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Imagescited by 152
- 2021Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Timecited by 140
- 2021Single-view robot pose and joint angle estimation via render & comparecited by 33
- 2022Interaction Replica: Tracking Human-Object Interaction and Scene Changes From Human Motioncited by 27
- 2022Hand-Object Interaction Image Generationcited by 19
- 2022Focal Length and Object Pose Estimation via Render and Comparecited by 17
- 2022Tracking and Reconstructing Hand Object Interactions from Point Cloud Sequences in the Wildcited by 17
- 2023ContactArt: Learning 3D Interaction Priors for Category-level Articulated Object and Hand Poses Estimationcited by 16
- 2023SHOWMe: Benchmarking Object-agnostic Hand-Object 3D Reconstructioncited by 16
- 2022TriHorn-Net: A Model for Accurate Depth-Based 3D Hand Pose Estimationcited by 7
12 most cited of 21 All 21 (JSON)
- 2025HOSt3R: Keypoint-free Hand-Object 3D Reconstruction from RGB images22 Aug 2025
- 2024Weakly-Supervised 3D Hand Reconstruction with Knowledge Prior and Uncertainty Guidance17 Jul 2024
- 2024HUP-3D: A 3D multi-view synthetic dataset for assisted-egocentric hand-ultrasound pose estimation12 Jul 2024
- 2023FocalPose++: Focal Length and Object Pose Estimation via Render and Compare15 Nov 2023
- 2023ShapeGraFormer: GraFormer-Based Network for Hand-Object Reconstruction from a Single Depth Map18 Oct 2023
- 2023SHOWMe: Benchmarking Object-agnostic Hand-Object 3D Reconstruction19 Sep 2023
- 2023ContactArt: Learning 3D Interaction Priors for Category-level Articulated Object and Hand Poses Estimation2 May 2023
- 2023Pushing the Envelope for Depth-Based Semi-Supervised 3D Hand Pose Estimation with Consistency Training27 Mar 2023
- 2023Tracking Fast by Learning Slow: An Event-based Speed Adaptive Hand Tracker Leveraging Knowledge in RGB Domain28 Feb 2023
- 2022Hand-Object Interaction Image Generation28 Nov 2022
- 2022Tracking and Reconstructing Hand Object Interactions from Point Cloud Sequences in the Wild24 Sep 2022
- 2022PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Images13 Jul 2022
12 newest of 21 All 21 (JSON)