mlsanityData to 7 Oct 2026

HandVoxNet++: 3D Hand Shape and Pose Estimation using Voxel-Based Neural Networks

arXiv 2107.01205 ↗First posted 2 Jul 2021cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

3D hand shape and pose estimation from a single depth map is a new and challenging computer vision problem with many applications. Existing methods addressing it directly regress hand meshes via 2D convolutional neural networks, which leads to artefacts due to perspective distortions in the images. To address the limitations of the existing methods, we develop HandVoxNet++, i.e., a voxel-based deep network with 3D and graph convolutions trained in a fully supervised manner. The input to our network is a 3D voxelized-depth-map-based on the truncated signed distance function (TSDF). HandVoxNet++ relies on two hand shape representations. The first one is the 3D voxelized grid of hand shape, which does not preserve the mesh topology and which is the most accurate representation. The second representation is the hand surface that preserves the mesh topology. We combine the advantages of both representations by aligning the hand surface to the voxelized hand shape either with a new neural Graph-Convolutions-based Mesh Registration (GCN-MeshReg) or classical segment-wise Non-Rigid Gravitational Approach (NRGA++) which does not rely on training data. In extensive evaluations on three public benchmarks, i.e., SynHand5M, depth-based HANDS19 challenge and HO-3D, the proposed HandVoxNet++ achieves state-of-the-art performance. In this journal extension of our previous approach presented at CVPR 2020, we gain 41.09% and 13.7% higher shape alignment accuracy on SynHand5M and HANDS19 datasets, respectively. Our method is ranked first on the HANDS19 challenge dataset (Task 1: Depth-Based 3D Hand Pose Estimation) at the moment of the submission of our results to the portal in August 2020.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 37 of its 67 references that are in the map. First posted on arXiv 2 Jul 2021.

Builds on

37 of 67 references.

References · 37

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  4. 2019Convolutional Mesh Regression for Single-Image Human Shape Reconstructioncited by 270
  5. 2018Generating 3D faces using Convolutional Mesh Autoencoderscited by 267
  6. 2017Integral Human Pose Regressioncited by 255
  7. 2017First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotationscited by 229
  8. 20193D Hand Shape and Pose Estimation from a Single RGB Imagecited by 174
  9. 2017GANerated Hands for Real-time 3D Hand Tracking from Monocular RGBcited by 147
  10. 2020Leveraging Photometric Consistency over Time for Sparsely Supervised Hand-Object Reconstructioncited by 139
  11. 2019End-to-end Hand Mesh Recovery from a Monocular RGB Imagecited by 124
  12. 2020HOPE-Net: A Graph-based Model for Hand-Object Pose Estimationcited by 112
25 more
  1. 2015Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Imagescited by 111
  2. 2020Monocular Real-time Hand Shape and Motion Capture using Multi-modal Datacited by 106
  3. 2017V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Mapcited by 102
  4. 2021Real-time Pose and Shape Reconstruction of Two Interacting Hands With a Single Depth Cameracited by 70
  5. 2019A2J: Anchor-to-Joint Regression Network for 3D Articulated Pose Estimation from a Single Depth Imagecited by 56
  6. 2020EventHands: Real-Time Neural 3D Hand Pose Estimation from an Event Streamcited by 48
  7. 2017BigHand2.2M Benchmark: Hand Pose Dataset and State of the Art Analysiscited by 46
  8. 2020SeqHAND:RGB-Sequence-Based 3D Hand Pose and Shape Estimationcited by 34
  9. 2020Measuring Generalisation to Unseen Viewpoints, Articulations, Shapes and Objects for 3D Hand Pose Estimation under Hand-Object Interactioncited by 24
  10. 2020HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose Estimation from a Single Depth Mapcited by 23
  11. 2021Differentiable Event Stream Simulator for Non-Rigid 3D Trackingcited by 23
  12. 2017Depth-Based 3D Hand Pose Estimation: From Current Achievements to Future Goalscited by 21
  13. 2019DEMEA: Deep Mesh Autoencoders for Non-Rigidly Deforming Objectscited by 20
  14. 2017DeepPrior++: Improving Fast and Accurate 3D Hand Pose Estimationcited by 19
  15. 2017Dense 3D Regression for Hand Pose Estimationcited by 18
  16. 2020AWR: Adaptive Weighting Regression for 3D Hand Pose Estimationcited by 17
  17. 2015Depth-based hand pose estimation: methods, data, and challengescited by 16
  18. 2019IsMo-GAN: Adversarial Learning for Monocular Non-Rigid 3D Reconstructioncited by 16
  19. 2017Feature Mapping for Learning Fast and Accurate 3D Pose Inference from Synthetic Imagescited by 15
  20. 2018HDM-Net: Monocular Non-Rigid 3D Reconstruction with Learned Deformation Modelcited by 15
  21. 2017Region Ensemble Network: Improving Convolutional Network for Hand Pose Estimationcited by 11
  22. 2019DispVoxNets: Non-Rigid Point Set Alignment with Supervised Learning Proxiescited by 8
  23. 2018DeepHPS: End-to-end Estimation of 3D Hand Pose and Shape by Learning from Synthetic Depthcited by 6
  24. 2018MURAUER: Mapping Unlabeled Real Data for Label AUstERitycited by 3
  25. 2017Simultaneous Hand Pose and Skeleton Bone-Lengths Estimation from a Single Depth Imagecited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 2; 2024: 3; 2025: 2; 2026: 2.

11 papers.

  1. 2024A survey of synthetic data augmentation methods in computer visioncited by 34
  2. 2022Scene-aware Egocentric 3D Human Pose Estimationcited by 30
  3. 2023High Fidelity 3D Hand Shape Reconstruction via Scalable Graph Frequency Decompositioncited by 18
  4. 2022THOR-Net: End-to-end Graformer-based Realistic Two Hands and Object Reconstruction with Self-supervisioncited by 16
  5. 2024Complementing Event Streams and RGB Frames for Hand Mesh Reconstructioncited by 14
  6. 2023ShapeGraFormer: GraFormer-Based Network for Hand-Object Reconstruction from a Single Depth Mapcited by 2
  7. 2024HandDAGT: A Denoising Adaptive Graph Transformer for 3D Hand Pose Estimationcited by 1
  8. 2025Geometry-Aware 3D Salient Object Detection Networkcited by 0
  9. 2025Enhancing Monocular 3D Hand Reconstruction with Learned Texture Priorscited by 0
  10. 2026HandMCM: Multi-modal Point Cloud-based Correspondence State Space Model for 3D Hand Pose Estimationcited by 0
  11. 2026HandMvNet: Real-Time 3D Hand Pose Estimation Using Multi-View Cross-Attention Fusioncited by 0