mlsanityData to 7 Oct 2026

DDF-HO: Hand-Held Object Reconstruction via Conditional Directed Distance Field

arXiv 2308.08231 ↗First posted 16 Aug 2023cs.CVcited by 16

Not on the map. Why?

Abstract (arXiv)

Reconstructing hand-held objects from a single RGB image is an important and challenging problem. Existing works utilizing Signed Distance Fields (SDF) reveal limitations in comprehensively capturing the complex hand-object interactions, since SDF is only reliable within the proximity of the target, and hence, infeasible to simultaneously encode local hand and object cues. To address this issue, we propose DDF-HO, a novel approach leveraging Directed Distance Field (DDF) as the shape representation. Unlike SDF, DDF maps a ray in 3D space, consisting of an origin and a direction, to corresponding DDF values, including a binary visibility signal determining whether the ray intersects the objects and a distance value measuring the distance from origin to target in the given direction. We randomly sample multiple rays and collect local to global geometric features for them by introducing a novel 2D ray-based feature aggregation scheme and a 3D intersection-aware hand pose embedding, combining 2D-3D features to model hand-object interactions. Extensive experiments on synthetic and real-world datasets demonstrate that DDF-HO consistently outperforms all baseline methods by a large margin, especially under Chamfer Distance, with about 80% leap forward. Codes are available at https://github.com/ZhangCYG/DDFHO.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 66 of its 82 references that are in the map. First posted on arXiv 16 Aug 2023.

Builds on

66 of 82 references.

References · 66

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  3. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  4. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  5. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
  6. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
  7. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
  8. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  9. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  10. 2018AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generationcited by 539
  11. 2019Learning joint reconstruction of hands and manipulated objectscited by 377
  12. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
38 more
  1. 2020Understanding Human Hands in Contact at Internet Scalecited by 281
  2. 2020Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Posecited by 248
  3. 2017Learning to Estimate 3D Hand Pose from Single RGB Imagescited by 234
  4. 2017First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotationscited by 229
  5. 2017Detecting and Recognizing Human-Object Interactionscited by 199
  6. 20193D Hand Shape and Pose from Images in the Wildcited by 190
  7. 2021CodeNeRF: Disentangled Neural Radiance Fields for Object Categoriescited by 183
  8. 2020ContactPose: A Dataset of Grasps with Object Contact and Hand Posecited by 180
  9. 2020Grasping Field: Learning Implicit Representations for Human Graspscited by 178
  10. 20193D Hand Shape and Pose Estimation from a Single RGB Imagecited by 174
  11. 2016Learning a Predictable and Generative Vector Representation for Objectscited by 173
  12. 2019Pix2Vox: Context-aware 3D Reconstruction from Single and Multi-view Imagescited by 164
  13. 2022RenderDiffusion: Image Diffusion for 3D Reconstruction, Inpainting and Generationcited by 162
  14. 2020Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wildcited by 152
  15. 2017GANerated Hands for Real-time 3D Hand Tracking from Monocular RGBcited by 147
  16. 2020Reconstructing Hand-Object Interactions in the Wildcited by 143
  17. 2021Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Timecited by 140
  18. 2019H+O: Unified Egocentric Recognition of 3D Hand-Object Poses and Interactionscited by 139
  19. 2021ContactOpt: Optimizing Contact to Improve Graspscited by 137
  20. 2017Learning Efficient Point Cloud Generation for Dense 3D Object Reconstructioncited by 133
  21. 2020Self-supervised Single-view 3D Reconstruction via Semantic Consistencycited by 127
  22. 2019End-to-end Hand Mesh Recovery from a Monocular RGB Imagecited by 124
  23. 2023UniDexGrasp: Universal Robotic Dexterous Grasping via Learning Diverse Proposal Generation and Goal-Conditioned Policycited by 117
  24. 2015Capturing Hands in Action using Discriminative Salient Points and Physics Simulationcited by 115
  25. 2020MeshSDF: Differentiable Iso-Surface Extractioncited by 112
  26. 2018Hand Pose Estimation via Latent 2.5D Heatmap Regressioncited by 111
  27. 2020Monocular Real-time Hand Shape and Motion Capture using Multi-modal Datacited by 106
  28. 2022GPV-Pose: Category-level Object Pose Estimation via Geometry-guided Point-wise Votingcited by 94
  29. 2020Shape and Viewpoint without Keypointscited by 92
  30. 2022What's in your hands? 3D Reconstruction of Generic Objects in Handscited by 90
  31. 2021SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimationcited by 88
  32. 2020HOI Analysis: Integrating and Decomposing Human-Object Interactioncited by 81
  33. 2020FrankMocap: Fast Monocular 3D Hand and Body Motion Capture by Regression and Integrationcited by 72
  34. 2021Real-time Pose and Shape Reconstruction of Two Interacting Hands With a Single Depth Cameracited by 70
  35. 2021Shelf-Supervised Mesh Prediction in the Wildcited by 65
  36. 2022AlignSDF: Pose-Aligned Signed Distance Fields for Hand-Object Reconstructioncited by 58
  37. 2014Category-Specific Object Reconstruction from a Single Imagecited by 56
  38. 2016Real-time Joint Tracking of a Hand Manipulating an Object from RGB-D Inputcited by 56

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 4; 2025: 3; 2026: 4.

16 papers.

Sort:

  1. 2025Open-Vocabulary Functional 3D Scene Graphs for Real-World Indoor Spacescited by 53
  2. 2023SecondPose: SE(3)-Consistent Dual-Stream Feature Fusion for Category-Level Pose Estimationcited by 52
  3. 2024BimArt: A Unified Approach for the Synthesis of 3D Bimanual Interaction with Articulated Objectscited by 29
  4. 2023HiPose: Hierarchical Binary Surface Encoding and Correspondence Pruning for RGB-D 6DoF Object Pose Estimationcited by 17
  5. 2023MOHO: Learning Single-view Hand-held Object Reconstruction with Multi-view Occlusion-Aware Supervisioncited by 12
  6. 2025HORT: Monocular Hand-held Objects Reconstruction with Transformerscited by 11
  7. 2023ShapeMatcher: Self-Supervised Joint Shape Canonicalization, Segmentation, Retrieval and Deformationcited by 7
  8. 2024KP-RED: Exploiting Semantic Keypoints for Joint 3D Shape Retrieval and Deformationcited by 6
  9. 2026ForeHOI: Feed-forward 3D Object Reconstruction from Daily Hand-Object Interaction Videoscited by 6
  10. 2025Occlusion-Aware 3D Hand-Object Pose Estimation with Masked AutoEncoderscited by 3
  11. 2023D-SCo: Dual-Stream Conditional Diffusion for Monocular Hand-Held Object Reconstructioncited by 2
  12. 2024Semantic-Rearrangement-Based Multi-Level Alignment for Domain Generalized Segmentationcited by 1
4 more
  1. 2026GenHOI: Generalized Hand-Object Pose Estimation with Occlusion Awarenesscited by 1
  2. 2026Hierarchical and Holistic Open-Vocabulary Functional 3D Scene Graphs for Indoor Spacescited by 1
  3. 2024Probabilistic Directed Distance Fields for Ray-Based Shape Representationscited by 0
  4. 2026Hand-Object Interaction in the Age of Large Foundation Models:Reconstruction, Generation, and Embodied Transfercited by 0