mlsanityData to 14 Oct 2026

UniHOPE: A Unified Approach for Hand-Only and Hand-Object Pose Estimation

arXiv 2503.13303 ↗First posted 17 Mar 2025cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

Estimating the 3D pose of hand and potential hand-held object from monocular images is a longstanding challenge. Yet, existing methods are specialized, focusing on either bare-hand or hand interacting with object. No method can flexibly handle both scenarios and their performance degrades when applied to the other scenario. In this paper, we propose UniHOPE, a unified approach for general 3D hand-object pose estimation, flexibly adapting both scenarios. Technically, we design a grasp-aware feature fusion module to integrate hand-object features with an object switcher to dynamically control the hand-object pose estimation according to grasping status. Further, to uplift the robustness of hand pose estimation regardless of object presence, we generate realistic de-occluded image pairs to train the model to learn object-induced hand occlusions, and formulate multi-level feature enhancement techniques for learning occlusion-invariant features. Extensive experiments on three commonly-used benchmarks demonstrate UniHOPE's SOTA performance in addressing hand-only and hand-object scenarios. Code will be released on https://github.com/JoyboyWang/UniHOPE_Pytorch.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 57 of its 72 references that are in the map. First posted on arXiv 17 Mar 2025.

Builds on

57 of 72 references.

References · 57

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  5. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  7. 2020Denoising Diffusion Implicit Modelscited by 9,781
  8. 2023Segment Anythingcited by 8,234
  9. 2023Visual Instruction Tuningcited by 7,869
  10. 2017Mask R-CNNcited by 7,277
  11. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  12. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
38 more
  1. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  2. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  3. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,430
  4. 2022Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Modelcited by 583
  5. 2023FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objectscited by 429
  6. 2019Learning joint reconstruction of hands and manipulated objectscited by 378
  7. 2021DexYCB: A Benchmark for Capturing Hand Grasping of Objectscited by 325
  8. 2020I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Imagecited by 274
  9. 2019FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Imagescited by 271
  10. 2020InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Imagecited by 245
  11. 2017Learning to Estimate 3D Hand Pose from Single RGB Imagescited by 234
  12. 20193D Hand Shape and Pose from Images in the Wildcited by 190
  13. 2020Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wildcited by 146
  14. 2020Reconstructing Hand-Object Interactions in the Wildcited by 144
  15. 2021Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Timecited by 141
  16. 2019End-to-end Hand Mesh Recovery from a Monocular RGB Imagecited by 124
  17. 2021Keypoint Transformer: Solving Joint Identification in Challenging Hands and Object Interactions for Accurate 3D Pose Estimationcited by 123
  18. 2019Pushing the Envelope for RGB-based Dense 3D Hand Pose Estimation via Neural Renderingcited by 115
  19. 2018Hand Pose Estimation via Latent 2.5D Heatmap Regressioncited by 111
  20. 2023Affordance Diffusion: Synthesizing Hand-Object Interactionscited by 110
  21. 2020Monocular Real-time Hand Shape and Motion Capture using Multi-modal Datacited by 106
  22. 2022HandOccNet: Occlusion-Robust 3D Hand Mesh Estimation Networkcited by 96
  23. 2022What's in your hands? 3D Reconstruction of Generic Objects in Handscited by 90
  24. 2021ArtiBoost: Boosting Articulated 3D Hand-Object Pose Estimation via Online Exploration and Synthesiscited by 76
  25. 2021MobRecon: Mobile-Friendly Hand Mesh Reconstruction from Monocular Imagecited by 74
  26. 2021Model-based 3D Hand Reconstruction via Self-Supervised Learningcited by 65
  27. 2021Camera-Space Hand Mesh Recovery via Semantic Aggregation and Adaptive 2D-1D Registrationcited by 59
  28. 2022AlignSDF: Pose-Aligned Signed Distance Fields for Hand-Object Reconstructioncited by 59
  29. 2022Collaborative Learning for Hand and Object Reconstruction with Attention-guided Graph Convolutioncited by 54
  30. 2023gSDF: Geometry-Driven Signed Distance Functions for 3D Hand-Object Reconstructioncited by 50
  31. 2024G-HOP: Generative Hand-Object Prior for Interaction Reconstruction and Grasp Synthesiscited by 43
  32. 2023HandRefiner: Refining Malformed Hands in Generated Images by Diffusion-based Conditional Inpaintingcited by 42
  33. 2024HanDiffuser: Text-to-Image Generation With Realistic Hand Appearancescited by 42
  34. 2024HOIDiffusion: Generating Realistic 3D Hand-Object Interaction Datacited by 37
  35. 2022Reconstructing Hand-Held Objects from Monocular Videocited by 35
  36. 2022Perspective Flow Aggregation for Data-Limited 6D Object Pose Estimationcited by 32
  37. 2021Hand Image Understanding via Deep Multi-Task Learningcited by 31
  38. 2024A Simple Baseline for Efficient Hand Mesh Reconstructioncited by 27

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 5.

6 papers.

  1. 2025Occlusion-Aware 3D Hand-Object Pose Estimation with Masked AutoEncoderscited by 3
  2. 2026EPS3D: End-to-End Feed-Forward 3D Panoptic Segmentationcited by 3
  3. 2026GenHOI: Generalized Hand-Object Pose Estimation with Occlusion Awarenesscited by 1
  4. 2026ComPose: When to Trust Hands for Object Pose Trackingcited by 0
  5. 2026High-Fidelity 4D Hand-Object Capture via Multi-View Spatiotemporal Tracking and Physics-Aware Gaussianscited by 0
  6. 2026CNS-Edit++: Category-Agnostic 3D Editing with Coupled Neural Shape Representationcited by 0