mlsanityData to 7 Oct 2026

Im2Hands: Learning Attentive Implicit Representation of Interacting Two-Hand Shapes

arXiv 2302.14348 ↗First posted 28 Feb 2023cs.CV cs.AIcited by 26

Not on the map. Why?

Abstract (arXiv)

We present Implicit Two Hands (Im2Hands), the first neural implicit representation of two interacting hands. Unlike existing methods on two-hand reconstruction that rely on a parametric hand model and/or low-resolution meshes, Im2Hands can produce fine-grained geometry of two hands with high hand-to-hand and hand-to-image coherency. To handle the shape complexity and interaction context between two hands, Im2Hands models the occupancy volume of two hands - conditioned on an RGB image and coarse 3D keypoints - by two novel attention-based modules responsible for (1) initial occupancy estimation and (2) context-aware occupancy refinement, respectively. Im2Hands first learns per-hand neural articulated occupancy in the canonical space designed for each hand using query-image attention. It then refines the initial two-hand occupancy in the posed space to enhance the coherency between the two hand shapes using query-anchor attention. In addition, we introduce an optional keypoint refinement module to enable robust two-hand shape estimation from predicted hand keypoints in a single-image reconstruction scenario. We experimentally demonstrate the effectiveness of Im2Hands on two-hand reconstruction in comparison to related methods, where ours achieves state-of-the-art results. Our code is publicly available at https://github.com/jyunlee/Im2Hands.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 36 of its 50 references that are in the map. First posted on arXiv 28 Feb 2023.

Builds on

36 of 50 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020Point Transformercited by 1,462
  4. 2021Volume Rendering of Neural Implicit Surfacescited by 990
  5. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  6. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 417
  7. 2019Learning joint reconstruction of hands and manipulated objectscited by 377
  8. 2020InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Imagecited by 244
  9. 2017Learning to Estimate 3D Hand Pose from Single RGB Imagescited by 234
  10. 2017Hand Keypoint Detection in Single Images using Multiview Bootstrappingcited by 231
  11. 2021Neural Articulated Radiance Fieldcited by 196
  12. 20193D Hand Shape and Pose from Images in the Wildcited by 190
24 more
  1. 2021SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networkscited by 181
  2. 2022Embodied Hands: Modeling and Capturing Hands and Bodies Togethercited by 179
  3. 2020Grasping Field: Learning Implicit Representations for Human Graspscited by 178
  4. 20193D Hand Shape and Pose Estimation from a Single RGB Imagecited by 174
  5. 2019NASA: Neural Articulated Shape Approximationcited by 149
  6. 2020Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wildcited by 146
  7. 2020Reconstructing Hand-Object Interactions in the Wildcited by 143
  8. 2020Leveraging Photometric Consistency over Time for Sparsely Supervised Hand-Object Reconstructioncited by 139
  9. 2019Pushing the Envelope for RGB-based Dense 3D Hand Pose Estimation via Neural Renderingcited by 115
  10. 2020HOPE-Net: A Graph-based Model for Hand-Object Pose Estimationcited by 112
  11. 2018Hand Pose Estimation via Latent 2.5D Heatmap Regressioncited by 111
  12. 2020Monocular Real-time Hand Shape and Motion Capture using Multi-modal Datacited by 106
  13. 2017V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Mapcited by 102
  14. 2020Weakly Supervised 3D Hand Pose Estimation via Biomechanical Constraintscited by 97
  15. 2022Interacting Attention Graph for Single Image Two-Hand Reconstructioncited by 87
  16. 2021Real-time Pose and Shape Reconstruction of Two Interacting Hands With a Single Depth Cameracited by 70
  17. 2022LISA: Learning Implicit Shape and Appearance of Handscited by 63
  18. 2022COAP: Compositional Articulated Occupancy of Peoplecited by 59
  19. 2021A Skeleton-Driven Neural Occupancy Representation for Articulated Handscited by 50
  20. 2021Learning to Disambiguate Strongly Interacting Hands via Probabilistic Per-pixel Part Segmentationcited by 40
  21. 2016Robust 3D Hand Pose Estimation in Single Depth Images: from Single-View CNN to Multi-View CNNscited by 29
  22. 2021Monocular 3D Reconstruction of Interacting Hands via Collision-Aware Factorized Refinementscited by 27
  23. 2021AIR-Nets: An Attention-Based Framework for Locally Conditioned Implicit Representationscited by 13
  24. 2019Dual Grid Net: hand mesh vertex regression from single depth mapscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 6; 2025: 8; 2026: 5.

26 papers.

Sort:

By area:

  1. 2023An Outlook into the Future of Egocentric Visioncited by 84
  2. 2023HOLD: Category-agnostic 3D Reconstruction of Interacting Hands and Objects from Videocited by 76
  3. 2024Dyn-HaMR: Recovering 4D Interacting Hand Motion from a Dynamic Cameracited by 50
  4. 2024Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objectscited by 35
  5. 2024InterHandGen: Two-Hand Interaction Generation via Cascaded Reverse Diffusioncited by 28
  6. 2024Dense Hand-Object(HO) GraspNet with Full Grasping Taxonomy and Dynamicscited by 15
  7. 2025MagicHOI: Leveraging 3D Priors for Accurate Hand-object Reconstruction from Short Monocular Video Clipscited by 15
  8. 2023FourierHandFlow: Neural 4D Hand Representation Using Fourier Query Flowcited by 10
  9. 2024BiTT: Bi-directional Texture Reconstruction of Interacting Two Hands from a Single Imagecited by 8
  10. 2025MPMAvatar: Learning 3D Gaussian Avatars with Accurate and Robust Physics-Based Dynamicscited by 8
  11. 2023Ins-HOI: Instance Aware Human-Object Interactions Recoverycited by 7
  12. 2026EgoGrasp: World-Space Hand-Object Interaction Estimation from Egocentric Videoscited by 7
14 more
  1. 2023CLIP-Hand3D: Exploiting 3D Hand Pose Estimation via Context-Aware Promptingcited by 5
  2. 2024Multi-hypotheses Conditioned Point Cloud Diffusion for 3D Human Reconstruction from Occluded Imagescited by 5
  3. 2025Advances in Radiance Field for Dynamic Scene: From Neural Field to Gaussian Fieldcited by 4
  4. 2025Advances in 4D Representation: Geometry, Motion, and Interactioncited by 4
  5. 2025Body-Hand Modality Expertized Networks with Cross-attention for Fine-grained Skeleton Action Recognitioncited by 3
  6. 2025SRHand: Super-Resolving Hand Images and 3D Shapes via View/Pose-aware Neural Image Representations and Explicit 3D Meshescited by 3
  7. 2023Pyramid Deep Fusion Network for Two-Hand Reconstruction from RGB-D Imagescited by 2
  8. 2023Monocular 3D Hand Mesh Recovery via Dual Noise Estimationcited by 0
  9. 2025LieHMR: Autoregressive Human Mesh Recovery with SO(3) Diffusioncited by 0
  10. 2025Cascaded Diffusion Framework for Probabilistic Coarse-to-Fine Hand Pose Estimationcited by 0
  11. 2026Gesture-Aware Pretraining and Token Fusion for 3D Hand Pose Estimationcited by 0
  12. 2026PhysHanDI: Physics-Based Reconstruction of Hand-Deformable Object Interactionscited by 0
  13. 2026Towards in-the-wild Egocentric 3D Hand-Object Pose Estimationcited by 0
  14. 2026Mudragen: Geometrically Supervised Generation of Interacting Two-Hand Mudras for Preserving Indian Classical Dance Heritagecited by 0