mlsanityData to 14 Oct 2026

Mask2Hand: Learning to Predict the 3D Hand Pose and Shape from Shadow

arXiv 2205.15553 ↗First posted 31 May 2022cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

We present a self-trainable method, Mask2Hand, which learns to solve the challenging task of predicting 3D hand pose and shape from a 2D binary mask of hand silhouette/shadow without additional manually-annotated data. Given the intrinsic camera parameters and the parametric hand model in the camera space, we adopt the differentiable rendering technique to project 3D estimations onto the 2D binary silhouette space. By applying a tailored combination of losses between the rendered silhouette and the input binary mask, we are able to integrate the self-guidance mechanism into our end-to-end optimization process for constraining global mesh registration and hand pose estimation. The experiments show that our method, which takes a single binary mask as the input, can achieve comparable prediction accuracy on both unaligned and aligned settings as state-of-the-art methods that require RGB or depth inputs. Our code is available at https://github.com/lijenchang/Mask2Hand.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 26 of its 44 references that are in the map. First posted on arXiv 31 May 2022.

Builds on

26 of 44 references.

References · 26

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 784
  3. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 418
  4. 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 405
  5. 2020I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Imagecited by 274
  6. 2019FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Imagescited by 271
  7. 20193D Hand Shape and Pose from Images in the Wildcited by 190
  8. 2020Monocular Expressive Body Regression through Body-Driven Attentioncited by 155
  9. 2021Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Timecited by 141
  10. 2020Leveraging Photometric Consistency over Time for Sparsely Supervised Hand-Object Reconstructioncited by 139
  11. 2021Advances in adversarial attacks and defenses in computer vision: A surveycited by 127
  12. 2019End-to-end Hand Mesh Recovery from a Monocular RGB Imagecited by 124
14 more
  1. 2020Differentiable Rendering: A Surveycited by 123
  2. 2015Capturing Hands in Action using Discriminative Salient Points and Physics Simulationcited by 115
  3. 2019Pushing the Envelope for RGB-based Dense 3D Hand Pose Estimation via Neural Renderingcited by 115
  4. 2020Monocular Real-time Hand Shape and Motion Capture using Multi-modal Datacited by 106
  5. 2017V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Mapcited by 102
  6. 2021Model-based 3D Hand Reconstruction via Self-Supervised Learningcited by 65
  7. 2021Camera-Space Hand Mesh Recovery via Semantic Aggregation and Adaptive 2D-1D Registrationcited by 59
  8. 2019A2J: Anchor-to-Joint Regression Network for 3D Articulated Pose Estimation from a Single Depth Imagecited by 56
  9. 2020EventHands: Real-Time Neural 3D Hand Pose Estimation from an Event Streamcited by 48
  10. 2020JGR-P2O: Joint Graph Reasoning based Pixel-to-Offset Prediction Network for 3D Hand Pose Estimation from a Single Depth Imagecited by 20
  11. 2020Exploiting Learnable Joint Groups for Hand Pose Estimationcited by 11
  12. 2021RealisticHands: A Hybrid Model for 3D Hand Reconstructioncited by 5
  13. 2018SolarGest: Ubiquitous and Battery-free Gesture Recognition using Solar Cellscited by 4
  14. 2019Silhouette-Net: 3D Hand Pose Estimation from Silhouettescited by 2

Built on by

Not cited yet.