mlsanityData to 14 Oct 2026

Using a single RGB frame for real time 3D hand pose estimation in the wild

arXiv 1712.03866 ↗First posted 11 Dec 2017cs.CVcited by 39

Not on the map. Why?

Abstract (arXiv)

We present a method for the real-time estimation of the full 3D pose of one or more human hands using a single commodity RGB camera. Recent work in the area has displayed impressive progress using RGBD input. However, since the introduction of RGBD sensors, there has been little progress for the case of monocular color input. We capitalize on the latest advancements of deep learning, combining them with the power of generative hand pose estimation techniques to achieve real-time monocular 3D hand pose estimation in unrestricted scenarios. More specifically, given an RGB image and the relevant camera calibration information, we employ a state-of-the-art detector to localize hands. Given a crop of a hand in the image, we run the pretrained network of OpenPose for hands to estimate the 2D location of hand joints. Finally, non-linear least-squares minimization fits a 3D model of the hand to the estimated 2D joint positions, recovering the 3D hand pose. Extensive experimental results provide comparison to the state of the art as well as qualitative assessment of the method in the wild.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). First posted on arXiv 11 Dec 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 16; 2022: 7; 2023: 8; 2024: 3; 2025: 4; 2026: 1.

39 papers.

Sort:

  1. 2021Keypoint Transformer: Solving Joint Identification in Challenging Hands and Object Interactions for Accurate 3D Pose Estimationcited by 123
  2. 2022What's in your hands? 3D Reconstruction of Generic Objects in Handscited by 90
  3. 2021Model-based 3D Hand Reconstruction via Self-Supervised Learningcited by 65
  4. 2022AlignSDF: Pose-Aligned Signed Distance Fields for Hand-Object Reconstructioncited by 59
  5. 2021A Skeleton-Driven Neural Occupancy Representation for Articulated Handscited by 50
  6. 2021Towards Accurate Alignment in Real-time 3D Hand-Mesh Reconstructioncited by 44
  7. 2021Learning to Disambiguate Strongly Interacting Hands via Probabilistic Per-pixel Part Segmentationcited by 40
  8. 2024EasyHOI: Unleashing the Power of Large Models for Reconstructing Hand-Object Interactions in the Wildcited by 36
  9. 2023Uncertainty-aware State Space Transformer for Egocentric 3D Hand Trajectory Forecastingcited by 34
  10. 2021Hand Image Understanding via Deep Multi-Task Learningcited by 31
  11. 2022Consistent 3D Hand Reconstruction in Video via self-supervised Learningcited by 28
  12. 2021RGB2Hands: Real-Time Tracking of 3D Hand Interactions from Monocular RGB Videocited by 23
27 more
  1. 2023DDF-HO: Hand-Held Object Reconstruction via Conditional Directed Distance Fieldcited by 17
  2. 2021HandTailor: Towards High-Precision Monocular 3D Hand Recoverycited by 15
  3. 2023MOHO: Learning Single-view Hand-held Object Reconstruction with Multi-view Occlusion-Aware Supervisioncited by 12
  4. 2025Survey on Hand Gesture Recognition from Visual Inputcited by 10
  5. 2025BIGS: Bimanual Category-agnostic Interaction Reconstruction from Monocular Videos via 3D Gaussian Splattingcited by 10
  6. 2021Multi-View Video-Based 3D Hand Pose Estimationcited by 6
  7. 2022End-to-end Weakly-supervised Single-stage Multiple 3D Hand Mesh Reconstruction from a Single RGB Imagecited by 6
  8. 2023Deep Learning Approaches for Seizure Video Analysis: A Reviewcited by 6
  9. 2021Im2Mesh GAN: Accurate 3D Hand Mesh Recovery from a Single RGB Imagecited by 5
  10. 20213D Hand Pose and Shape Estimation from RGB Images for Keypoint-Based Hand Gesture Recognitioncited by 5
  11. 2021Dynamic Iterative Refinement for Efficient 3D Hand Pose Estimationcited by 3
  12. 2023D-SCo: Dual-Stream Conditional Diffusion for Monocular Hand-Held Object Reconstructioncited by 3
  13. 2023HandyPriors: Physically Consistent Perception of Hand-Object Interactions with Differentiable Priorscited by 3
  14. 2026HGGT: Robust and Flexible 3D Hand Mesh Reconstruction from Uncalibrated Imagescited by 3
  15. 2021A hybrid classification-regression approach for 3D hand pose estimation using graph convolutional networkscited by 2
  16. 2021Multi-view Image-based Hand Geometry Refinement using Differentiable Monte Carlo Ray Tracingcited by 2
  17. 2023Pyramid Deep Fusion Network for Two-Hand Reconstruction from RGB-D Imagescited by 2
  18. 2024Learning to Score Sign Language with Two-stage Methodcited by 2
  19. 2021Semi-Supervised 3D Hand Shape and Pose Estimation with Label Propagationcited by 1
  20. 2022Learning Sequential Contexts using Transformer for 3D Hand Pose Estimationcited by 1
  21. 2022End-to-End 3D Hand Pose Estimation from Stereo Camerascited by 1
  22. 2024Enhancing Sign Language Teaching: A Mixed Reality Approach for Immersive Learning and Multi-Dimensional Feedbackcited by 1
  23. 2021Parallel mesh reconstruction streams for pose estimation of interacting handscited by 0
  24. 2022LG-Hand: Advancing 3D Hand Pose Estimation with Locally and Globally Kinematic Knowledgecited by 0
  25. 2023Self-supervised Optimization of Hand Pose Estimation using Anatomical Features and Iterative Learningcited by 0
  26. 2025Testing Human-Hand Segmentation on In-Distribution and Out-of-Distribution Data in Human-Robot Interactions Using a Deep Ensemble Modelcited by 0
  27. 2025Monocular 3D Hand Pose Estimation with Implicit Camera Alignmentcited by 0