# HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose Estimation from a Single Depth Map

arXiv: https://arxiv.org/abs/2004.01588 · HTML: https://mlsanity.com/paper/2004.01588/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Apr 2020.

## Built on by (most cited first, 23 of 23)

- [TOCH: Spatio-Temporal Object-to-Hand Correspondence for Motion Refinement](https://mlsanity.com/paper/2205.07982/) (2022)
- [Towards Accurate Alignment in Real-time 3D Hand-Mesh Reconstruction](https://mlsanity.com/paper/2109.01723/) (2021)
- [Hand Image Understanding via Deep Multi-Task Learning](https://mlsanity.com/paper/2107.11646/) (2021)
- [Scene-aware Egocentric 3D Human Pose Estimation](https://mlsanity.com/paper/2212.11684/) (2022)
- [3D Pose Estimation of Two Interacting Hands from a Monocular Event Camera](https://mlsanity.com/paper/2312.14157/) (2023)
- [THOR-Net: End-to-end Graformer-based Realistic Two Hands and Object Reconstruction with Self-supervision](https://mlsanity.com/paper/2210.13853/) (2022)
- [HandTailor: Towards High-Precision Monocular 3D Hand Recovery](https://mlsanity.com/paper/2102.09244/) (2021)
- [HandVoxNet++: 3D Hand Shape and Pose Estimation using Voxel-Based Neural Networks](https://mlsanity.com/paper/2107.01205/) (2021)
- [Physics-aware Hand-object Interaction Denoising](https://mlsanity.com/paper/2405.11481/) (2024)
- [TriHorn-Net: A Model for Accurate Depth-Based 3D Hand Pose Estimation](https://mlsanity.com/paper/2206.07117/) (2022)
- [Local and Global Point Cloud Reconstruction for 3D Hand Pose Estimation](https://mlsanity.com/paper/2112.06389/) (2021)
- [3D Hand Pose and Shape Estimation from RGB Images for Keypoint-Based Hand Gesture Recognition](https://mlsanity.com/paper/2109.13879/) (2021)
- [HandFlow: Quantifying View-Dependent 3D Ambiguity in Two-Hand Reconstruction with Normalizing Flow](https://mlsanity.com/paper/2210.01692/) (2022)
- [Image-free Domain Generalization via CLIP for 3D Hand Pose Estimation](https://mlsanity.com/paper/2210.16788/) (2022)
- [TSCom-Net: Coarse-to-Fine 3D Textured Shape Completion Network](https://mlsanity.com/paper/2208.08768/) (2022)
- [ShapeGraFormer: GraFormer-Based Network for Hand-Object Reconstruction from a Single Depth Map](https://mlsanity.com/paper/2310.11811/) (2023)
- [SiMA-Hand: Boosting 3D Hand-Mesh Reconstruction by Single-to-Multi-View Adaptation](https://mlsanity.com/paper/2402.01389/) (2024)
- [Pushing the Envelope for Depth-Based Semi-Supervised 3D Hand Pose Estimation with Consistency Training](https://mlsanity.com/paper/2303.15147/) (2023)
- [Intelligent Generation of Graphical Game Assets: A Conceptual Framework and Systematic Review of the State of the Art](https://mlsanity.com/paper/2311.10129/) (2023)
- [EvHand-FPV: Efficient Event-Based 3D Hand Tracking from First-Person View](https://mlsanity.com/paper/2509.13883/) (2025)
- [Enhancing Monocular 3D Hand Reconstruction with Learned Texture Priors](https://mlsanity.com/paper/2508.09629/) (2025)
- [Fast-HaMeR: Boosting Hand Mesh Reconstruction using Knowledge Distillation](https://mlsanity.com/paper/2603.16444/) (2026)
- [HandMvNet: Real-Time 3D Hand Pose Estimation Using Multi-View Cross-Attention Fusion](https://mlsanity.com/paper/2608.20093/) (2026)

## Abstract (arXiv)

3D hand shape and pose estimation from a single depth map is a new and challenging computer vision problem with many applications. The state-of-the-art methods directly regress 3D hand meshes from 2D depth images via 2D convolutional neural networks, which leads to artefacts in the estimations due to perspective distortions in the images. In contrast, we propose a novel architecture with 3D convolutions trained in a weakly-supervised manner. The input to our method is a 3D voxelized depth map, and we rely on two hand shape representations. The first one is the 3D voxelized grid of the shape which is accurate but does not preserve the mesh topology and the number of mesh vertices. The second representation is the 3D hand surface which is less accurate but does not suffer from the limitations of the first representation. We combine the advantages of these two representations by registering the hand surface to the voxelized hand shape. In the extensive experiments, the proposed approach improves over the state of the art by 47.8% on the SynHand5M dataset. Moreover, our augmentation policy for voxelized depth maps further enhances the accuracy of 3D hand pose estimation on real data. Our method produces visually more reasonable and realistic hand shapes on NYU and BigHand2.2M datasets compared to the existing approaches.