mlsanityData to 14 Oct 2026

HOISDF: Constraining 3D Hand-Object Pose Estimation with Global Signed Distance Fields

arXiv 2402.17062 ↗First posted 26 Feb 2024cs.CVcited by 14

Not on the map. Why?

Abstract (arXiv)

Human hands are highly articulated and versatile at handling objects. Jointly estimating the 3D poses of a hand and the object it manipulates from a monocular camera is challenging due to frequent occlusions. Thus, existing methods often rely on intermediate 3D shape representations to increase performance. These representations are typically explicit, such as 3D point clouds or meshes, and thus provide information in the direct surroundings of the intermediate hand pose estimate. To address this, we introduce HOISDF, a Signed Distance Field (SDF) guided hand-object pose estimation network, which jointly exploits hand and object SDFs to provide a global, implicit representation over the complete reconstruction volume. Specifically, the role of the SDFs is threefold: equip the visual encoder with implicit shape information, help to encode hand-object interactions, and guide the hand and object pose regression via SDF-based sampling and by augmenting the feature representations. We show that HOISDF achieves state-of-the-art results on hand-object pose estimation benchmarks (DexYCB and HO3Dv2). Code is available at https://github.com/amathislab/HOISDF

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 45 of its 59 references that are in the map. First posted on arXiv 26 Feb 2024.

Builds on

45 of 59 references.

References · 45

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  6. 2020End-to-End Object Detection with Transformerscited by 7,414
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  8. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  9. 2021Transformers in Vision: A Surveycited by 1,013
  10. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 429
  11. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 418
  12. 2018PVNet: Pixel-wise Voting Network for 6DoF Pose Estimationcited by 385
33 more
  1. 2020INeRF: Inverting Neural Radiance Fields for Pose Estimationcited by 382
  2. 2019Learning joint reconstruction of hands and manipulated objectscited by 378
  3. 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 327
  4. 2021DexYCB: A Benchmark for Capturing Hand Grasping of Objectscited by 325
  5. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
  6. 2019FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Imagescited by 271
  7. 2020Neural Unsigned Distance Fields for Implicit Function Learningcited by 265
  8. 2017Learning to Estimate 3D Hand Pose from Single RGB Imagescited by 234
  9. 2019PVN3D: A Deep Point-wise 3D Keypoints Voting Network for 6DoF Pose Estimationcited by 204
  10. 2022Embodied Hands: Modeling and Capturing Hands and Bodies Togethercited by 181
  11. 2020Grasping Field: Learning Implicit Representations for Human Graspscited by 179
  12. 2021FrankMocap: A Monocular 3D Whole-Body Pose Estimation System via Regression and Integrationcited by 169
  13. 2020Neural-Pull: Learning Signed Distance Functions from Point Clouds by Learning to Pull Space onto Surfacescited by 147
  14. 2021Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Timecited by 141
  15. 2020Leveraging Photometric Consistency over Time for Sparsely Supervised Hand-Object Reconstructioncited by 139
  16. 2021Keypoint Transformer: Solving Joint Identification in Challenging Hands and Object Interactions for Accurate 3D Pose Estimationcited by 123
  17. 2022HandOccNet: Occlusion-Robust 3D Hand Mesh Estimation Networkcited by 96
  18. 2022CHORE: Contact, Human and Object REconstruction from a single RGB imagecited by 95
  19. 2022What's in your hands? 3D Reconstruction of Generic Objects in Handscited by 90
  20. 2023EPro-PnP: Generalized End-to-End Probabilistic Perspective-n-Points for Monocular Object Pose Estimationcited by 79
  21. 2021ArtiBoost: Boosting Articulated 3D Hand-Object Pose Estimation via Online Exploration and Synthesiscited by 76
  22. 2021MobRecon: Mobile-Friendly Hand Mesh Reconstruction from Monocular Imagecited by 74
  23. 2022AlignSDF: Pose-Aligned Signed Distance Fields for Hand-Object Reconstructioncited by 59
  24. 2021Towards unconstrained joint hand-object reconstruction from RGB videoscited by 58
  25. 2019A2J: Anchor-to-Joint Regression Network for 3D Articulated Pose Estimation from a Single Depth Imagecited by 56
  26. 2022Collaborative Learning for Hand and Object Reconstruction with Attention-guided Graph Convolutioncited by 54
  27. 2023gSDF: Geometry-Driven Signed Distance Functions for 3D Hand-Object Reconstructioncited by 50
  28. 2019Deep learning tools for the measurement of animal behavior in neurosciencecited by 44
  29. 2020Joint Hand-object 3D Reconstruction from a Single Image with Cross-branch Feature Fusioncited by 39
  30. 2022Interacting Hand-Object Pose Estimation via Dense Mutual Attentioncited by 22
  31. 2018Point-to-Pose Voting based Hand Pose Estimation using Residual Permutation Equivariant Layercited by 21
  32. 2020Recent Advances in 3D Object and Hand Pose Estimationcited by 9
  33. 2022DMAP: a Distributed Morphological Attention Policy for Learning to Locomote with a Changing Bodycited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 7; 2026: 3.

14 papers.

Sort:

By area:

  1. 2024Hamba: Single-view 3D Hand Reconstruction with Graph-guided Bi-Scanning Mambacited by 45
  2. 2024Deep Learning-Based Object Pose Estimation: A Comprehensive Surveycited by 40
  3. 2024InterTrack: Tracking Human Object Interaction without Object Templatescited by 23
  4. 2025HORT: Monocular Hand-held Objects Reconstruction with Transformerscited by 11
  5. 2025Diffusion-based 3D Hand Motion Recovery with Intuitive Physicscited by 5
  6. 2025Towards Dynamic 3D Reconstruction of Hand-Instrument Interaction in Ophthalmic Surgerycited by 3
  7. 2025Occlusion-Aware 3D Hand-Object Pose Estimation with Masked AutoEncoderscited by 3
  8. 2025Estimating 2D Keypoints of Surgical Tools Using Vision-Language Models with Low-Rank Adaptationcited by 1
  9. 2025Interaction-Aware 4D Gaussian Splatting for Dynamic Hand-Object Interaction Reconstructioncited by 1
  10. 2026GenHOI: Generalized Hand-Object Pose Estimation with Occlusion Awarenesscited by 1
  11. 2026Hand-Object Interaction in the Age of Large Foundation Models:Reconstruction, Generation, and Embodied Transfercited by 1
  12. 2024HandS3C: 3D Hand Mesh Reconstruction with State Space Spatial Channel Attention from RGB imagescited by 0
2 more
  1. 2025VPHO: Joint Visual-Physical Cue Learning and Aggregation for Hand-Object Pose Estimationcited by 0
  2. 2026LEXIS: LatEnt ProXimal Interaction Signatures for 3D HOI from an Imagecited by 0