mlsanityData to 7 Oct 2026

A Skeleton-Driven Neural Occupancy Representation for Articulated Hands

arXiv 2109.11399 ↗First posted 23 Sep 2021cs.CVcited by 50

Not on the map. Why?

Abstract (arXiv)

We present Hand ArticuLated Occupancy (HALO), a novel representation of articulated hands that bridges the advantages of 3D keypoints and neural implicit surfaces and can be used in end-to-end trainable architectures. Unlike existing statistical parametric hand models (e.g.~MANO), HALO directly leverages 3D joint skeleton as input and produces a neural occupancy volume representing the posed hand surface. The key benefits of HALO are (1) it is driven by 3D key points, which have benefits in terms of accuracy and are easier to learn for neural networks than the latent hand-model parameters; (2) it provides a differentiable volumetric occupancy representation of the posed hand; (3) it can be trained end-to-end, allowing the formulation of losses on the hand surface that benefit the learning of 3D keypoints. We demonstrate the applicability of HALO to the task of conditional generation of hands that grasp 3D objects. The differentiable nature of HALO is shown to improve the quality of the synthesized hands both in terms of physical plausibility and user preference.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Builds on 57 of its 71 references that are in the map. First posted on arXiv 23 Sep 2021.

Builds on

57 of 71 references.

References · 57

  1. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  2. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  3. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  4. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  5. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  6. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  7. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  8. 2020Convolutional Occupancy Networkscited by 760
  9. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  10. 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 397
  11. 2019Learning joint reconstruction of hands and manipulated objectscited by 377
  12. 2021DexYCB: A Benchmark for Capturing Hand Grasping of Objectscited by 320
38 more
  1. 2020I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Imagecited by 274
  2. 2020ARCH: Animatable Reconstruction of Clothed Humanscited by 257
  3. 2020Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Posecited by 248
  4. 2020InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Imagecited by 244
  5. 2019Learning Shape Templates with Structured Implicit Functionscited by 242
  6. 20196-DOF GraspNet: Variational Grasp Generation for Object Manipulationcited by 237
  7. 2017Learning to Estimate 3D Hand Pose from Single RGB Imagescited by 234
  8. 2017Hand Keypoint Detection in Single Images using Multiview Bootstrappingcited by 231
  9. 2017First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotationscited by 229
  10. 20193D Hand Shape and Pose from Images in the Wildcited by 190
  11. 2021SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networkscited by 181
  12. 2020ContactPose: A Dataset of Grasps with Object Contact and Hand Posecited by 180
  13. 2019DIST: Rendering Deep Implicit Signed Distance Function with Differentiable Sphere Tracingcited by 179
  14. 2020Grasping Field: Learning Implicit Representations for Human Graspscited by 178
  15. 20193D Hand Shape and Pose Estimation from a Single RGB Imagecited by 174
  16. 2019CvxNet: Learnable Convex Decompositioncited by 173
  17. 2017GANerated Hands for Real-time 3D Hand Tracking from Monocular RGBcited by 147
  18. 2020Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wildcited by 146
  19. 2019ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imagingcited by 145
  20. 2021Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Timecited by 140
  21. 2019H+O: Unified Egocentric Recognition of 3D Hand-Object Poses and Interactionscited by 139
  22. 2020Leveraging Photometric Consistency over Time for Sparsely Supervised Hand-Object Reconstructioncited by 139
  23. 2021ContactOpt: Optimizing Contact to Improve Graspscited by 137
  24. 2019Generating 3D People in Scenes without Peoplecited by 127
  25. 2019End-to-end Hand Mesh Recovery from a Monocular RGB Imagecited by 124
  26. 2019Pushing the Envelope for RGB-based Dense 3D Hand Pose Estimation via Neural Renderingcited by 115
  27. 2020HOPE-Net: A Graph-based Model for Hand-Object Pose Estimationcited by 112
  28. 2018Hand Pose Estimation via Latent 2.5D Heatmap Regressioncited by 111
  29. 2020LoopReg: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for 3D Human Mesh Registrationcited by 108
  30. 2021LEAP: Learning Articulated Occupancy of Peoplecited by 107
  31. 2020Monocular Real-time Hand Shape and Motion Capture using Multi-modal Datacited by 106
  32. 2019Learning to Infer Implicit Surfaces without 3D Supervisioncited by 103
  33. 2017V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Mapcited by 102
  34. 2019Efficient Learning on Point Clouds with Basis Point Setscited by 101
  35. 2020Weakly Supervised 3D Hand Pose Estimation via Biomechanical Constraintscited by 97
  36. 2018Cross-modal Deep Variational Hand Pose Estimationcited by 89
  37. 2021Real-time Pose and Shape Reconstruction of Two Interacting Hands With a Single Depth Cameracited by 70
  38. 2020DeepHandMesh: A Weakly-supervised Deep Encoder-Decoder Framework for High-fidelity Hand Mesh Modelingcited by 49

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 6; 2023: 14; 2024: 18; 2025: 6; 2026: 3.

50 papers.

Sort:

By area:

  1. 2021Neural Fields in Visual Computing and Beyondcited by 539
  2. 2021D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactionscited by 109
  3. 2024TACO: Benchmarking Generalizable Bimanual Tool-ACtion-Object Understandingcited by 85
  4. 2021MobRecon: Mobile-Friendly Hand Mesh Reconstruction from Monocular Imagecited by 74
  5. 2022AlignSDF: Pose-Aligned Signed Distance Fields for Hand-Object Reconstructioncited by 58
  6. 2022CaDeX: Learning Canonical Deformation Coordinate Space for Dynamic Surface Representation via Neural Homeomorphismcited by 56
  7. 2023ArtiGrasp: Physically Plausible Synthesis of Bi-Manual Dexterous Grasping and Articulationcited by 53
  8. 2023ContactGen: Generative Contact Modeling for Grasp Generationcited by 52
  9. 2023gSDF: Geometry-Driven Signed Distance Functions for 3D Hand-Object Reconstructioncited by 49
  10. 2022Hand Avatar: Free-Pose Hand Animation and Rendering from Monocular Videocited by 37
  11. 2023UGG: Unified Generative Graspingcited by 30
  12. 2024SemGrasp: Semantic Grasp Generation via Language Aligned Discretizationcited by 29
38 more
  1. 2022HARP: Personalized Hand Reconstruction from a Monocular RGB Videocited by 28
  2. 2024InterHandGen: Two-Hand Interaction Generation via Cascaded Reverse Diffusioncited by 28
  3. 2023Im2Hands: Learning Attentive Implicit Representation of Interacting Two-Hand Shapescited by 26
  4. 2023GRIP: Generating Interaction Poses Using Spatial Cues and Latent Consistencycited by 26
  5. 2023RelightableHands: Efficient Neural Relighting of Articulated Hand Modelscited by 21
  6. 2024ManiDext: Hand-Object Manipulation Synthesis via Continuous Correspondence Embeddings and Residual-Guided Diffusioncited by 21
  7. 2024GEARS: Local Geometry-aware Hand-object Interaction Synthesiscited by 20
  8. 2024OHTA: One-shot Hand Avatar via Data-driven Implicit Priorscited by 18
  9. 2024URHand: Universal Relightable Handscited by 17
  10. 2023Novel-view Synthesis and Pose Estimation for Hand-Object Interaction from Sparse Viewscited by 16
  11. 2024Dense Hand-Object(HO) GraspNet with Full Grasping Taxonomy and Dynamicscited by 15
  12. 2023FourierHandFlow: Neural 4D Hand Representation Using Fourier Query Flowcited by 10
  13. 2023CHORD: Category-level Hand-held Object Reconstruction via Shape Deformationcited by 10
  14. 2023Spectral Graphormer: Spectral Graph-based Transformer for Egocentric Two-Hand Reconstruction using Multi-View Color Imagescited by 9
  15. 2022Stability-driven Contact Reconstruction From Monocular Color Imagescited by 8
  16. 2022UV-Based 3D Hand-Object Reconstruction with Grasp Optimizationcited by 8
  17. 2024BiTT: Bi-directional Texture Reconstruction of Interacting Two Hands from a Single Imagecited by 8
  18. 2023Human 3D Avatar Modeling with Implicit Neural Representation: A Brief Surveycited by 7
  19. 2023Ins-HOI: Instance Aware Human-Object Interactions Recoverycited by 7
  20. 2024Text2Grasp: Grasp synthesis by text prompts of object grasping partscited by 7
  21. 2024HandDGP: Camera-Space Hand Mesh Prediction with Differentiable Global Positioningcited by 7
  22. 2024FastGrasp: Efficient Grasp Synthesis with Diffusioncited by 7
  23. 20243D Whole-body Grasp Synthesis with Directional Controllabilitycited by 6
  24. 2023PHRIT: Parametric Hand Representation with Implicit Templatecited by 5
  25. 20243D Visibility-aware Generalizable Neural Radiance Fields for Interacting Handscited by 4
  26. 2024MLPHand: Real Time Multi-View 3D Hand Mesh Reconstruction via MLP Modelingcited by 3
  27. 2024Decomposed Vector-Quantized Variational Autoencoder for Human Grasp Generationcited by 3
  28. 2025JGHand: Joint-Driven Animatable Hand Avater via 3D Gaussian Splattingcited by 3
  29. 2025PALM: A Dataset and Baseline for Learning Multi-subject Hand Priorcited by 3
  30. 2025FunHOI: Annotation-Free 3D Hand-Object Interaction Generation via Functional Text Guidancecited by 2
  31. 2025HandOcc: NeRF-based Hand Rendering with Occupancy Networkscited by 2
  32. 2024Two Hands Are Better Than One: Resolving Hand to Hand Intersections via Occupancy Networkscited by 1
  33. 2024A Versatile and Differentiable Hand-Object Interaction Representationcited by 1
  34. 2025Task-Oriented Human Grasp Synthesis via Context- and Task-Aware Diffuserscited by 1
  35. 2025SynHLMA:Synthesizing Hand Language Manipulation for Articulated Object with Discrete Human Object Interaction Representationcited by 1
  36. 2026AffordGrasp: Cross-Modal Diffusion for Affordance-Aware Grasp Synthesiscited by 1
  37. 2026Hand-Object Interaction in the Age of Large Foundation Models:Reconstruction, Generation, and Embodied Transfercited by 0
  38. 2026VolCo: Volumetric Contact for High-Fidelity Human Grasp Generationcited by 0