mlsanityData to 14 Oct 2026

Neural Voting Field for Camera-Space 3D Hand Pose Estimation

arXiv 2305.04328 ↗First posted 7 May 2023cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

We present a unified framework for camera-space 3D hand pose estimation from a single RGB image based on 3D implicit representation. As opposed to recent works, most of which first adopt holistic or pixel-level dense regression to obtain relative 3D hand pose and then follow with complex second-stage operations for 3D global root or scale recovery, we propose a novel unified 3D dense regression scheme to estimate camera-space 3D hand pose via dense 3D point-wise voting in camera frustum. Through direct dense modeling in 3D domain inspired by Pixel-aligned Implicit Functions for 3D detailed reconstruction, our proposed Neural Voting Field (NVF) fully models 3D dense local evidence and hand global geometry, helping to alleviate common 2D-to-3D ambiguities. Specifically, for a 3D query point in camera frustum and its pixel-aligned image feature, NVF, represented by a Multi-Layer Perceptron, regresses: (i) its signed distance to the hand surface; (ii) a set of 4D offset vectors (1D voting weight and 3D directional vector to each hand joint). Following a vote-casting scheme, 4D offset vectors from near-surface points are selected to calculate the 3D hand joint coordinates by a weighted average. Experiments demonstrate that NVF outperforms existing state-of-the-art algorithms on FreiHAND dataset for camera-space 3D hand pose estimation. We also adapt NVF to the classic task of root-relative 3D hand pose estimation, for which NVF also obtains state-of-the-art results on HO3D dataset.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 51 of its 65 references that are in the map. First posted on arXiv 7 May 2023.

Builds on

51 of 65 references.

References · 51

  1. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  2. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  3. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  4. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 915
  5. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  6. 2020Implicit Geometric Regularization for Learning Shapescited by 710
  7. 2020PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitizationcited by 521
  8. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 418
  9. 2019SAL: Sign Agnostic Learning of Shapes from Raw Datacited by 390
  10. 2019Learning joint reconstruction of hands and manipulated objectscited by 378
  11. 2019DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstructioncited by 333
  12. 2016Convolutional Pose Machinescited by 325
38 more
  1. 2020I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Imagecited by 274
  2. 2019FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Imagescited by 271
  3. 2021Mesh Graphormercited by 265
  4. 2020ARCH: Animatable Reconstruction of Clothed Humanscited by 257
  5. 2020Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Posecited by 248
  6. 2017Learning to Estimate 3D Hand Pose from Single RGB Imagescited by 234
  7. 2020PaMIR: Parametric Model-Conditioned Implicit Representation for Image-based Human Reconstructioncited by 231
  8. 2016Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Posecited by 221
  9. 2020MetaSDF: Meta-learning Signed Distance Functionscited by 214
  10. 2019Texture Fields: Learning Texture Representations in Function Spacecited by 210
  11. 20193D Hand Shape and Pose from Images in the Wildcited by 190
  12. 2020Grasping Field: Learning Implicit Representations for Human Graspscited by 179
  13. 20193D Hand Shape and Pose Estimation from a Single RGB Imagecited by 174
  14. 2020Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wildcited by 146
  15. 2021Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Timecited by 141
  16. 2020Leveraging Photometric Consistency over Time for Sparsely Supervised Hand-Object Reconstructioncited by 139
  17. 2019Camera Distance-aware Top-down Approach for 3D Multi-person Pose Estimation from a Single RGB Imagecited by 132
  18. 2019End-to-end Hand Mesh Recovery from a Monocular RGB Imagecited by 124
  19. 2021Keypoint Transformer: Solving Joint Identification in Challenging Hands and Object Interactions for Accurate 3D Pose Estimationcited by 123
  20. 2019Pushing the Envelope for RGB-based Dense 3D Hand Pose Estimation via Neural Renderingcited by 115
  21. 2020SALD: Sign Agnostic Learning with Derivativescited by 115
  22. 2018Hand Pose Estimation via Latent 2.5D Heatmap Regressioncited by 111
  23. 2022HandOccNet: Occlusion-Robust 3D Hand Mesh Estimation Networkcited by 96
  24. 2018Cross-modal Deep Variational Hand Pose Estimationcited by 89
  25. 2019How do neural networks see depth in single images?cited by 82
  26. 2021ArtiBoost: Boosting Articulated 3D Hand-Object Pose Estimation via Online Exploration and Synthesiscited by 76
  27. 2021MobRecon: Mobile-Friendly Hand Mesh Reconstruction from Monocular Imagecited by 74
  28. 2021Model-based 3D Hand Reconstruction via Self-Supervised Learningcited by 65
  29. 2021Camera-Space Hand Mesh Recovery via Semantic Aggregation and Adaptive 2D-1D Registrationcited by 59
  30. 2021Towards unconstrained joint hand-object reconstruction from RGB videoscited by 58
  31. 2019A2J: Anchor-to-Joint Regression Network for 3D Articulated Pose Estimation from a Single Depth Imagecited by 56
  32. 2022Collaborative Learning for Hand and Object Reconstruction with Attention-guided Graph Convolutioncited by 54
  33. 2021I2UV-HandNet: Image-to-UV Prediction Network for Accurate and High-fidelity 3D Hand Mesh Modelingcited by 52
  34. 2018Point-to-Pose Voting based Hand Pose Estimation using Residual Permutation Equivariant Layercited by 21
  35. 2022Neural Correspondence Field for Object Pose Estimationcited by 21
  36. 2020JGR-P2O: Joint Graph Reasoning based Pixel-to-Offset Prediction Network for 3D Hand Pose Estimation from a Single Depth Imagecited by 20
  37. 2017Dense 3D Regression for Hand Pose Estimationcited by 18
  38. 2021Fully Understanding Generic Objects: Modeling, Segmentation, and Reconstructioncited by 16

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 1; 2026: 2.

6 papers.

  1. 2024HandDGP: Camera-Space Hand Mesh Prediction with Differentiable Global Positioningcited by 7
  2. 2025UniHOPE: A Unified Approach for Hand-Only and Hand-Object Pose Estimationcited by 6
  3. 20243D Hand Reconstruction via Aggregating Intra and Inter Graphs Guided by Prior Knowledge for Hand-Object Interaction Scenariocited by 2
  4. 20243D Hand Mesh Recovery from Monocular RGB in Camera Spacecited by 0
  5. 2026SesaHand: Enhancing 3D Hand Reconstruction via Controllable Generation with Semantic and Structural Alignmentcited by 0
  6. 2026MHE-Former: Multi-Hypothesis Transformers via Entropy Maximization for 3D Mesh Recoverycited by 0