mlsanityData to 14 Oct 2026

Semantics2Hands: Transferring Hand Motion Semantics between Avatars

arXiv 2308.05920 ↗First posted 11 Aug 2023cs.CV cs.GR cs.MMcited by 0

Not on the map. Why?

Abstract (arXiv)

Human hands, the primary means of non-verbal communication, convey intricate semantics in various scenarios. Due to the high sensitivity of individuals to hand motions, even minor errors in hand motions can significantly impact the user experience. Real applications often involve multiple avatars with varying hand shapes, highlighting the importance of maintaining the intricate semantics of hand motions across the avatars. Therefore, this paper aims to transfer the hand motion semantics between diverse avatars based on their respective hand models. To address this problem, we introduce a novel anatomy-based semantic matrix (ASM) that encodes the semantics of hand motions. The ASM quantifies the positions of the palm and other joints relative to the local frame of the corresponding joint, enabling precise retargeting of hand motions. Subsequently, we obtain a mapping function from the source ASM to the target hand joint rotations by employing an anatomy-based semantics reconstruction network (ASRN). We train the ASRN using a semi-supervised learning strategy on the Mixamo and InterHand2.6M datasets. We evaluate our method in intra-domain and cross-domain hand motion retargeting tasks. The qualitative and quantitative results demonstrate the significant superiority of our ASRN over the state-of-the-arts.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 47 references that are in the map. First posted on arXiv 11 Aug 2023.

Builds on

18 of 47 references.

References · 18

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 405
  5. 2019Learning joint reconstruction of hands and manipulated objectscited by 378
  6. 2020InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Imagecited by 245
  7. 2022Embodied Hands: Modeling and Capturing Hands and Bodies Togethercited by 181
  8. 2020Grasping Field: Learning Implicit Representations for Human Graspscited by 179
  9. 2020CPF: Learning a Contact Potential Field to Model the Hand-Object Interactioncited by 128
  10. 2020Skeleton-Aware Networks for Deep Motion Retargetingcited by 124
  11. 2021Synergies Between Affordance and Geometry: 6-DoF Grasp Detection via Implicit Representationscited by 108
  12. 2013Data-Driven Grasp Synthesis - A Surveycited by 106
6 more
  1. 2018Neural Kinematic Networks for Unsupervised Motion Retargettingcited by 88
  2. 2022TOCH: Spatio-Temporal Object-to-Hand Correspondence for Motion Refinementcited by 70
  3. 2022Signing at Scale: Learning to Co-Articulate Signs for Large-Scale Photo-Realistic Sign Language Productioncited by 61
  4. 2023Skinned Motion Retargeting with Residual Perception of Motion Semantics & Geometrycited by 35
  5. 2020Body2Hands: Learning to Infer 3D Hands from Conversational Gesture Body Dynamicscited by 34
  6. 2023Diverse 3D Hand Gesture Prediction from Body Dynamics by Bilateral Hand Disentanglementcited by 18

Built on by

Not cited yet.