mlsanityData to 14 Oct 2026

Dense Hand-Object(HO) GraspNet with Full Grasping Taxonomy and Dynamics

arXiv 2409.04033 ↗First posted 6 Sep 2024cs.CVcited by 15

Not on the map. Why?

Abstract (arXiv)

Existing datasets for 3D hand-object interaction are limited either in the data cardinality, data variations in interaction scenarios, or the quality of annotations. In this work, we present a comprehensive new training dataset for hand-object interaction called HOGraspNet. It is the only real dataset that captures full grasp taxonomies, providing grasp annotation and wide intraclass variations. Using grasp taxonomies as atomic actions, their space and time combinatorial can represent complex hand activities around objects. We select 22 rigid objects from the YCB dataset and 8 other compound objects using shape and size taxonomies, ensuring coverage of all hand grasp configurations. The dataset includes diverse hand shapes from 99 participants aged 10 to 74, continuous video frames, and a 1.5M RGB-Depth of sparse frames with annotations. It offers labels for 3D hand and object meshes, 3D keypoints, contact maps, and grasp labels. Accurate hand and object 3D meshes are obtained by fitting the hand parametric model (MANO) and the hand implicit function (HALO) to multi-view RGBD frames, with the MoCap system only for objects. Note that HALO fitting does not require any parameter tuning, enabling scalability to the dataset's size with comparable accuracy to MANO. We evaluate HOGraspNet on relevant tasks: grasp classification and 3D hand pose estimation. The result shows performance variations based on grasp type and object class, indicating the potential importance of the interaction space captured by our dataset. The provided data aims at learning universal shape priors or foundation models for 3D hand-object interaction. Our dataset and code are available at https://hograspnet2024.github.io/.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 70 of its 99 references that are in the map. First posted on arXiv 6 Sep 2024.

Builds on

70 of 99 references.

References · 70

  1. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,161
  2. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  3. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  4. 2019MediaPipe: A Framework for Building Perception Pipelinescited by 583
  5. 2020Rescaling Egocentric Visioncited by 540
  6. 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 405
  7. 2019Learning joint reconstruction of hands and manipulated objectscited by 378
  8. 2021DexYCB: A Benchmark for Capturing Hand Grasping of Objectscited by 325
  9. 2022ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulationcited by 312
  10. 2023Reconstructing Hands in 3D with Transformerscited by 306
  11. 2019Resolving 3D Human Pose Ambiguities with 3D Scene Constraintscited by 295
  12. 2019FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Imagescited by 271
38 more
  1. 2018Generating 3D faces using Convolutional Mesh Autoencoderscited by 268
  2. 2021Mesh Graphormercited by 265
  3. 2021H2O: Two Hands Manipulating Objects for First Person Interaction Recognitioncited by 254
  4. 2022BEHAVE: Dataset and Method for Tracking Human Object Interactionscited by 249
  5. 2020InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Imagecited by 245
  6. 2017Learning to Estimate 3D Hand Pose from Single RGB Imagescited by 234
  7. 2017Hand Keypoint Detection in Single Images using Multiview Bootstrappingcited by 231
  8. 2017First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotationscited by 230
  9. 2021AGORA: Avatars in Geography Optimized for Regression Analysiscited by 205
  10. 2017Single-Shot Multi-Person 3D Pose Estimation From Monocular RGBcited by 195
  11. 2020ContactPose: A Dataset of Grasps with Object Contact and Hand Posecited by 183
  12. 2022Capturing and Inferring Dense Full-Body Human-Scene Contactcited by 180
  13. 2019ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imagingcited by 147
  14. 2020Reconstructing Hand-Object Interactions in the Wildcited by 144
  15. 2021Putting People in their Place: Monocular Regression of 3D People in Depthcited by 142
  16. 2019H+O: Unified Egocentric Recognition of 3D Hand-Object Poses and Interactionscited by 141
  17. 2021Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Timecited by 141
  18. 2020Leveraging Photometric Consistency over Time for Sparsely Supervised Hand-Object Reconstructioncited by 139
  19. 2021ContactOpt: Optimizing Contact to Improve Graspscited by 139
  20. 2021EgoBody: Human Body Shape and Motion of Interacting People from Head-Mounted Devicescited by 139
  21. 2022OakInk: A Large-scale Knowledge Repository for Understanding Hand-Object Interactioncited by 131
  22. 2020CPF: Learning a Contact Potential Field to Model the Hand-Object Interactioncited by 128
  23. 2021Keypoint Transformer: Solving Joint Identification in Challenging Hands and Object Interactions for Accurate 3D Pose Estimationcited by 123
  24. 2015Capturing Hands in Action using Discriminative Salient Points and Physics Simulationcited by 115
  25. 2020HOPE-Net: A Graph-based Model for Hand-Object Pose Estimationcited by 112
  26. 2022InterCap: Joint Markerless 3D Tracking of Humans and Objects in Interactioncited by 93
  27. 2022Interacting Attention Graph for Single Image Two-Hand Reconstructioncited by 87
  28. 2023Hi4D: 4D Instance Segmentation of Close Human Interactioncited by 87
  29. 2017Real-time Hand Tracking under Occlusion from an Egocentric RGB-D Sensorcited by 84
  30. 2022Full-Body Articulated Human-Object Interactioncited by 78
  31. 2021Multi-Person Extreme Motion Predictioncited by 75
  32. 2021DeepMultiCap: Performance Capture of Multiple Characters Using Sparse Multiview Camerascited by 67
  33. 2022AlignSDF: Pose-Aligned Signed Distance Fields for Hand-Object Reconstructioncited by 59
  34. 20193DPeople: Modeling the Geometry of Dressed Humanscited by 58
  35. 2021Towards unconstrained joint hand-object reconstruction from RGB videoscited by 58
  36. 2021Human Hands as Probes for Interactive Object Understandingcited by 52
  37. 2021A Skeleton-Driven Neural Occupancy Representation for Articulated Handscited by 50
  38. 2023gSDF: Geometry-Driven Signed Distance Functions for 3D Hand-Object Reconstructioncited by 50

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 6; 2026: 8.

15 papers.

Sort:

By area:

  1. 2024HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videoscited by 129
  2. 2025Open-Vocabulary Functional 3D Scene Graphs for Real-World Indoor Spacescited by 54
  3. 2026SHOW3D: Capturing Scenes of 3D Hands and Objects in the Wildcited by 4
  4. 2025Reconstructing Objects along Hand Interaction Timelines in Egocentric Videocited by 3
  5. 2025Affordance-Guided Diffusion Prior for 3D Hand Reconstructioncited by 2
  6. 2026Event6D: Event-based Novel Object 6D Pose Trackingcited by 2
  7. 2025Joint Learning of Pose Regression and Denoising Diffusion with Score Scaling Sampling for Category-level 6D Pose Estimationcited by 1
  8. 2026Hierarchical and Holistic Open-Vocabulary Functional 3D Scene Graphs for Indoor Spacescited by 1
  9. 2026Hand-Object Interaction in the Age of Large Foundation Models:Reconstruction, Generation, and Embodied Transfercited by 1
  10. 2025LieHMR: Autoregressive Human Mesh Recovery with SO(3) Diffusioncited by 0
  11. 2025Cascaded Diffusion Framework for Probabilistic Coarse-to-Fine Hand Pose Estimationcited by 0
  12. 2026PhysHanDI: Physics-Based Reconstruction of Hand-Deformable Object Interactionscited by 0
3 more
  1. 2026Dynamic Full-body Motion Agent with Object Interaction via Blending Pre-trained Modular Controllerscited by 0
  2. 2026VEPHand: View-Efficient Photometric Hand Performance Capture at Scalecited by 0
  3. 2026PhotoHOI: Synthesizing 3D Hand-Object Interactions from a Single RGB Photographcited by 0