mlsanityData to 7 Oct 2026

gSDF: Geometry-Driven Signed Distance Functions for 3D Hand-Object Reconstruction

arXiv 2304.11970 ↗First posted 24 Apr 2023cs.CVcited by 49

Not on the map. Why?

Abstract (arXiv)

Signed distance functions (SDFs) is an attractive framework that has recently shown promising results for 3D shape reconstruction from images. SDFs seamlessly generalize to different shape resolutions and topologies but lack explicit modelling of the underlying 3D geometry. In this work, we exploit the hand structure and use it as guidance for SDF-based shape reconstruction. In particular, we address reconstruction of hands and manipulated objects from monocular RGB images. To this end, we estimate poses of hands and objects and use them to guide 3D reconstruction. More specifically, we predict kinematic chains of pose transformations and align SDFs with highly-articulated hand poses. We improve the visual features of 3D points with geometry alignment and further leverage temporal information to enhance the robustness to occlusion and motion blurs. We conduct extensive experiments on the challenging ObMan and DexYCB benchmarks and demonstrate significant improvements of the proposed method over the state of the art.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Builds on 56 of its 71 references that are in the map. First posted on arXiv 24 Apr 2023.

Builds on

56 of 71 references.

References · 56

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  4. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  5. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  6. 2021ViViT: A Video Vision Transformercited by 1,610
  7. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
  8. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
  9. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  10. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  11. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 625
  12. 2018AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generationcited by 539
38 more
  1. 2016Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervisioncited by 493
  2. 2019Learning joint reconstruction of hands and manipulated objectscited by 377
  3. 2021DexYCB: A Benchmark for Capturing Hand Grasping of Objectscited by 320
  4. 2020HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimationcited by 304
  5. 2019What Do Single-view 3D Reconstruction Networks Learn?cited by 259
  6. 2017Integral Human Pose Regressioncited by 255
  7. 2017Learning to Estimate 3D Hand Pose from Single RGB Imagescited by 234
  8. 2016Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Posecited by 220
  9. 20193D Hand Shape and Pose from Images in the Wildcited by 190
  10. 2022Embodied Hands: Modeling and Capturing Hands and Bodies Togethercited by 179
  11. 2020Grasping Field: Learning Implicit Representations for Human Graspscited by 178
  12. 2017GANerated Hands for Real-time 3D Hand Tracking from Monocular RGBcited by 147
  13. 2020Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wildcited by 146
  14. 2020Reconstructing Hand-Object Interactions in the Wildcited by 143
  15. 2019H+O: Unified Egocentric Recognition of 3D Hand-Object Poses and Interactionscited by 139
  16. 2020Leveraging Photometric Consistency over Time for Sparsely Supervised Hand-Object Reconstructioncited by 139
  17. 2022OakInk: A Large-scale Knowledge Repository for Understanding Hand-Object Interactioncited by 129
  18. 2020CPF: Learning a Contact Potential Field to Model the Hand-Object Interactioncited by 127
  19. 2021Keypoint Transformer: Solving Joint Identification in Challenging Hands and Object Interactions for Accurate 3D Pose Estimationcited by 122
  20. 2019Pushing the Envelope for RGB-based Dense 3D Hand Pose Estimation via Neural Renderingcited by 115
  21. 2018Hand Pose Estimation via Latent 2.5D Heatmap Regressioncited by 111
  22. 2018Ordinal Depth Supervision for 3D Human Pose Estimationcited by 107
  23. 2017V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Mapcited by 102
  24. 2022What's in your hands? 3D Reconstruction of Generic Objects in Handscited by 90
  25. 2022Interacting Attention Graph for Single Image Two-Hand Reconstructioncited by 87
  26. 2021ArtiBoost: Boosting Articulated 3D Hand-Object Pose Estimation via Online Exploration and Synthesiscited by 76
  27. 2021Real-time Pose and Shape Reconstruction of Two Interacting Hands With a Single Depth Cameracited by 70
  28. 2021Deep Regression on Manifolds: A 3D Rotation Case Studycited by 67
  29. 2022LISA: Learning Implicit Shape and Appearance of Handscited by 63
  30. 2021Camera-Space Hand Mesh Recovery via Semantic Aggregation and Adaptive 2D-1D Registrationcited by 59
  31. 2022AlignSDF: Pose-Aligned Signed Distance Fields for Hand-Object Reconstructioncited by 58
  32. 2021Towards unconstrained joint hand-object reconstruction from RGB videoscited by 57
  33. 2016Real-time Joint Tracking of a Hand Manipulating an Object from RGB-D Inputcited by 56
  34. 2019A2J: Anchor-to-Joint Regression Network for 3D Articulated Pose Estimation from a Single Depth Imagecited by 56
  35. 2022Collaborative Learning for Hand and Object Reconstruction with Attention-guided Graph Convolutioncited by 54
  36. 2021A Skeleton-Driven Neural Occupancy Representation for Articulated Handscited by 50
  37. 20223D Interacting Hand Pose Estimation by Hand De-occlusion and Removalcited by 36
  38. 2019Single Image 3D Hand Reconstruction with Mesh Convolutionscited by 32

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 20; 2025: 13; 2026: 7.

49 papers.

Sort:

By area:

  1. 2023An Outlook into the Future of Egocentric Visioncited by 84
  2. 2023HOLD: Category-agnostic 3D Reconstruction of Interacting Hands and Objects from Videocited by 76
  3. 2023Diffusion-Guided Reconstruction of Everyday Hand-Object Interaction Clipscited by 73
  4. 2024G-HOP: Generative Hand-Object Prior for Interaction Reconstruction and Grasp Synthesiscited by 43
  5. 2024Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objectscited by 35
  6. 2024ViViDex: Learning Vision-based Dexterous Manipulation from Human Videoscited by 33
  7. 2024EasyHOI: Unleashing the Power of Large Models for Reconstructing Hand-Object Interactions in the Wildcited by 33
  8. 2024HO-Cap: A Capture System and Dataset for 3D Reconstruction and Pose Tracking of Hand-Object Interactioncited by 25
  9. 2024Reconstructing Hand-Held Objects in 3D from Images and Videoscited by 23
  10. 2024Vinci: A Real-time Embodied Smart Assistant based on Egocentric Vision-Language Modelcited by 19
  11. 2024Dense Hand-Object(HO) GraspNet with Full Grasping Taxonomy and Dynamicscited by 15
  12. 2025MagicHOI: Leveraging 3D Priors for Accurate Hand-object Reconstruction from Short Monocular Video Clipscited by 15
37 more
  1. 2024HOISDF: Constraining 3D Hand-Object Pose Estimation with Global Signed Distance Fieldscited by 14
  2. 20233D Reconstruction of Objects in Hands without Real World 3D Supervisioncited by 12
  3. 2023MOHO: Learning Single-view Hand-held Object Reconstruction with Multi-view Occlusion-Aware Supervisioncited by 12
  4. 2025HORT: Monocular Hand-held Objects Reconstruction with Transformerscited by 11
  5. 2023Learning Explicit Contact for Implicit Reconstruction of Hand-held Objects from Monocular Imagescited by 10
  6. 2024PEAR: Phrase-Based Hand-Object Interaction Anticipationcited by 10
  7. 2025BIGS: Bimanual Category-agnostic Interaction Reconstruction from Monocular Videos via 3D Gaussian Splattingcited by 10
  8. 2024MS-MANO: Enabling Hand Pose Tracking with Biomechanical Constraintscited by 9
  9. 2025Follow My Hold: Hand-Object Interaction Reconstruction through Geometric Guidancecited by 8
  10. 2023DiffHand: End-to-End Hand Mesh Reconstruction via Diffusion Modelscited by 7
  11. 2023Ins-HOI: Instance Aware Human-Object Interactions Recoverycited by 7
  12. 2024FastGrasp: Efficient Grasp Synthesis with Diffusioncited by 7
  13. 2024Hand-Object Interaction Controller (HOIC): Deep Reinforcement Learning for Reconstructing Interactions with Physicscited by 6
  14. 2025UniHOPE: A Unified Approach for Hand-Only and Hand-Object Pose Estimationcited by 6
  15. 2026ForeHOI: Feed-forward 3D Object Reconstruction from Daily Hand-Object Interaction Videoscited by 6
  16. 2024Sparse multi-view hand-object reconstruction for unseen environmentscited by 4
  17. 2024OCC-MLLM:Empowering Multimodal Large Language Model For the Understanding of Occluded Objectscited by 3
  18. 2024Posture-Informed Muscular Force Learning for Robust Hand Pressure Estimationcited by 3
  19. 2025Occlusion-Aware 3D Hand-Object Pose Estimation with Masked AutoEncoderscited by 3
  20. 2025HOSt3R: Keypoint-free Hand-Object 3D Reconstruction from RGB imagescited by 3
  21. 2026ArtHOI: Taming Foundation Models for Monocular 4D Reconstruction of Hand-Articulated-Object Interactionscited by 3
  22. 2023D-SCo: Dual-Stream Conditional Diffusion for Monocular Hand-Held Object Reconstructioncited by 2
  23. 2024Stereo Hand-Object Reconstruction for Human-to-Robot Handovercited by 2
  24. 2025Collaborative Learning for 3D Hand-Object Reconstruction and Compositional Action Recognition from Egocentric RGB Videos Using Superquadricscited by 2
  25. 2025FunHOI: Annotation-Free 3D Hand-Object Interaction Generation via Functional Text Guidancecited by 2
  26. 2026GHOST: Fast Category-agnostic Hand-Object Interaction Reconstruction from RGB Videos using Gaussian Splattingcited by 2
  27. 2024A Versatile and Differentiable Hand-Object Interaction Representationcited by 1
  28. 2024OCC-MLLM-Alpha:Empowering Multi-modal Large Language Model for the Understanding of Occluded Objects with Self-Supervised Test-Time Learningcited by 1
  29. 2026GenHOI: Generalized Hand-Object Pose Estimation with Occlusion Awarenesscited by 1
  30. 2024VR-based generation of photorealistic synthetic data for training hand-object tracking modelscited by 0
  31. 2025PartHOI: Part-based Hand-Object Interaction Transfer via Generalized Cylinderscited by 0
  32. 2025TIGeR: Text-Instructed Generation and Refinement for Template-Free Hand-Object Interactioncited by 0
  33. 2025PROD: Palpative Reconstruction of Deformable Objects through Elastostatic Signed Distance Functionscited by 0
  34. 2025VPHO: Joint Visual-Physical Cue Learning and Aggregation for Hand-Object Pose Estimationcited by 0
  35. 2026PhysHanDI: Physics-Based Reconstruction of Hand-Deformable Object Interactionscited by 0
  36. 2026TextHOI-3D: Text-to-3D Hand-Object Interaction via Discrete Multi-View Generation and Joint Mesh Optimizationcited by 0
  37. 2026Hand-Object Interaction in the Age of Large Foundation Models:Reconstruction, Generation, and Embodied Transfercited by 0