mlsanityData to 14 Oct 2026

Model-based 3D Hand Reconstruction via Self-Supervised Learning

arXiv 2103.11703 ↗First posted 22 Mar 2021cs.CVcited by 65

Not on the map. Why?

Abstract (arXiv)

Reconstructing a 3D hand from a single-view RGB image is challenging due to various hand configurations and depth ambiguity. To reliably reconstruct a 3D hand from a monocular image, most state-of-the-art methods heavily rely on 3D annotations at the training stage, but obtaining 3D annotations is expensive. To alleviate reliance on labeled training data, we propose S2HAND, a self-supervised 3D hand reconstruction network that can jointly estimate pose, shape, texture, and the camera viewpoint. Specifically, we obtain geometric cues from the input image through easily accessible 2D detected keypoints. To learn an accurate hand reconstruction model from these noisy geometric cues, we utilize the consistency between 2D and 3D representations and propose a set of novel losses to rationalize outputs of the neural network. For the first time, we demonstrate the feasibility of training an accurate 3D hand reconstruction network without relying on manual annotations. Our experiments show that the proposed method achieves comparable performance with recent fully-supervised methods while using fewer supervision data.

Key facts. Cited by 65 papers in the map (data through 14 Oct 2026). Builds on 29 of its 53 references that are in the map. First posted on arXiv 22 Mar 2021.

Builds on

29 of 53 references.

References · 29

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  3. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,162
  4. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 915
  5. 2017Neural 3D Mesh Renderercited by 437
  6. 2019Learning joint reconstruction of hands and manipulated objectscited by 378
  7. 2020I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Imagecited by 274
  8. 2019FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Imagescited by 271
  9. 2017Integral Human Pose Regressioncited by 256
  10. 2017Learning to Estimate 3D Hand Pose from Single RGB Imagescited by 234
  11. 20193D Hand Shape and Pose from Images in the Wildcited by 190
  12. 2019Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wildcited by 180
17 more
  1. 20193D Hand Shape and Pose Estimation from a Single RGB Imagecited by 174
  2. 2017MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstructioncited by 163
  3. 2020Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wildcited by 146
  4. 2020Leveraging Photometric Consistency over Time for Sparsely Supervised Hand-Object Reconstructioncited by 139
  5. 2019End-to-end Hand Mesh Recovery from a Monocular RGB Imagecited by 124
  6. 2019Pushing the Envelope for RGB-based Dense 3D Hand Pose Estimation via Neural Renderingcited by 115
  7. 2018Hand Pose Estimation via Latent 2.5D Heatmap Regressioncited by 111
  8. 2020Monocular Real-time Hand Shape and Motion Capture using Multi-modal Datacited by 106
  9. 2020Weakly Supervised 3D Hand Pose Estimation via Biomechanical Constraintscited by 97
  10. 2020Shape and Viewpoint without Keypointscited by 92
  11. 2017Self-supervised Multi-level Face Model Learning for Monocular Reconstruction at over 250 Hzcited by 91
  12. 2017Using a single RGB frame for real time 3D hand pose estimation in the wildcited by 39
  13. 2020Joint Hand-object 3D Reconstruction from a Single Image with Cross-branch Feature Fusioncited by 39
  14. 2020SeqHAND:RGB-Sequence-Based 3D Hand Pose and Shape Estimationcited by 34
  15. 2016Robust 3D Hand Pose Estimation in Single Depth Images: from Single-View CNN to Multi-View CNNscited by 29
  16. 2020Measuring Generalisation to Unseen Viewpoints, Articulations, Shapes and Objects for 3D Hand Pose Estimation under Hand-Object Interactioncited by 24
  17. 2019Self-supervised Learning of Detailed 3D Face Reconstructioncited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 7; 2022: 10; 2023: 15; 2024: 16; 2025: 11; 2026: 6.

65 papers.

Sort:

By area:

  1. 2021FrankMocap: A Monocular 3D Whole-Body Pose Estimation System via Regression and Integrationcited by 169
  2. 2022MixSTE: Seq2seq Mixed Spatio-Temporal Encoder for 3D Human Pose Estimation in Videocited by 159
  3. 2021MobRecon: Mobile-Friendly Hand Mesh Reconstruction from Monocular Imagecited by 74
  4. 2024Hamba: Single-view 3D Hand Reconstruction with Graph-guided Bi-Scanning Mambacited by 45
  5. 2022Hand Avatar: Free-Pose Hand Animation and Rendering from Monocular Videocited by 37
  6. 2021Lightweight Multi-person Total Motion Capture Using Sparse Multi-view Camerascited by 33
  7. 2023A Probabilistic Attention Model with Occlusion-aware Texture Regression for 3D Hand Reconstruction from a Single RGB Imagecited by 32
  8. 2022Consistent 3D Hand Reconstruction in Video via self-supervised Learningcited by 28
  9. 2022HARP: Personalized Hand Reconstruction from a Monocular RGB Videocited by 28
  10. 2022Neural Shape Deformation Priorscited by 27
  11. 2022H3WB: Human3.6M 3D WholeBody Dataset and Benchmarkcited by 23
  12. 2024Controlling the World by Sleight of Handcited by 19
38 more
  1. 2023High Fidelity 3D Hand Shape Reconstruction via Scalable Graph Frequency Decompositioncited by 18
  2. 2023Reconstructing Interacting Hands with Interaction Prior from Monocular Imagescited by 18
  3. 2024OHTA: One-shot Hand Avatar via Data-driven Implicit Priorscited by 18
  4. 2022Efficient Annotation and Learning for 3D Hand Pose Estimation: A Surveycited by 17
  5. 2023EvHandPose: Event-based 3D Hand Pose Estimation with Sparse Supervisioncited by 17
  6. 2024Complementing Event Streams and RGB Frames for Hand Mesh Reconstructioncited by 14
  7. 2024HHMR: Holistic Hand Mesh Recovery by Enhancing the Multimodal Controllability of Graph Diffusion Modelscited by 14
  8. 2024HandBooster: Boosting 3D Hand-Mesh Reconstruction by Conditional Synthesis and Sampling of Hand-Object Interactionscited by 13
  9. 20233D Points Splatting for Real-Time Dynamic Hand Reconstructioncited by 10
  10. 2024HandOS: 3D Hand Reconstruction in One Stagecited by 9
  11. 2022UV-Based 3D Hand-Object Reconstruction with Grasp Optimizationcited by 8
  12. 2023MeMaHand: Exploiting Mesh-Mano Interaction for Single Image Two-Hand Reconstructioncited by 8
  13. 2024BiTT: Bi-directional Texture Reconstruction of Interacting Two Hands from a Single Imagecited by 8
  14. 2023HaMuCo: Hand Pose Estimation via Multiview Collaborative Self-Supervised Learningcited by 7
  15. 2024HandDGP: Camera-Space Hand Mesh Prediction with Differentiable Global Positioningcited by 7
  16. 2025SiMHand: Mining Similar Hands for Large-Scale 3D Hand Pose Pre-trainingcited by 7
  17. 2021Local and Global Point Cloud Reconstruction for 3D Hand Pose Estimationcited by 6
  18. 2022End-to-end Weakly-supervised Single-stage Multiple 3D Hand Mesh Reconstruction from a Single RGB Imagecited by 6
  19. 2023Neural Voting Field for Camera-Space 3D Hand Pose Estimationcited by 6
  20. 2024Multi-view Hand Reconstruction with a Point-Embedded Transformercited by 6
  21. 2025UniHOPE: A Unified Approach for Hand-Only and Hand-Object Pose Estimationcited by 6
  22. 2021RealisticHands: A Hybrid Model for 3D Hand Reconstructioncited by 5
  23. 2023PHRIT: Parametric Hand Representation with Implicit Templatecited by 5
  24. 2023CLIP-Hand3D: Exploiting 3D Hand Pose Estimation via Context-Aware Promptingcited by 5
  25. 2024Weakly-Supervised 3D Hand Reconstruction with Knowledge Prior and Uncertainty Guidancecited by 5
  26. 2024MaskHand: Generative Masked Modeling for Robust Hand Mesh Reconstruction in the Wildcited by 5
  27. 2025Diffusion-based 3D Hand Motion Recovery with Intuitive Physicscited by 5
  28. 2023Overcoming the Trade-off Between Accuracy and Plausibility in 3D Hand Shape Reconstructioncited by 4
  29. 2024Learning Interaction-aware 3D Gaussian Splatting for One-shot Hand Avatarscited by 3
  30. 2025SRHand: Super-Resolving Hand Images and 3D Shapes via View/Pose-aware Neural Image Representations and Explicit 3D Meshescited by 3
  31. 2026Towards Egocentric 3D Hand Pose Estimation in Unseen Domainscited by 3
  32. 2021Multi-view Image-based Hand Geometry Refinement using Differentiable Monte Carlo Ray Tracingcited by 2
  33. 2023HiFiHR: Enhancing 3D Hand Reconstruction from a Single Image via High-Fidelity Texturecited by 2
  34. 20243D Hand Reconstruction via Aggregating Intra and Inter Graphs Guided by Prior Knowledge for Hand-Object Interaction Scenariocited by 2
  35. 2021Semi-Supervised 3D Hand Shape and Pose Estimation with Label Propagationcited by 1
  36. 2023Exploiting Spatial-Temporal Context for Interacting Hand Reconstruction on Monocular RGB Videocited by 1
  37. 20243D WholeBody Pose Estimation based on Semantic Graph Attention Network and Distance Informationcited by 1
  38. 2025HandSCS: Structural Coordinate Space for Animatable Hand Gaussian Splattingcited by 1

50 most cited of 65 All 65 (JSON)