mlsanityData to 7 Oct 2026

Hand-Centric Motion Refinement for 3D Hand-Object Interaction via Hierarchical Spatial-Temporal Modeling

arXiv 2401.15987 ↗First posted 29 Jan 2024cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

Hands are the main medium when people interact with the world. Generating proper 3D motion for hand-object interaction is vital for applications such as virtual reality and robotics. Although grasp tracking or object manipulation synthesis can produce coarse hand motion, this kind of motion is inevitably noisy and full of jitter. To address this problem, we propose a data-driven method for coarse motion refinement. First, we design a hand-centric representation to describe the dynamic spatial-temporal relation between hands and objects. Compared to the object-centric representation, our hand-centric representation is straightforward and does not require an ambiguous projection process that converts object-based prediction into hand motion. Second, to capture the dynamic clues of hand-object interaction, we propose a new architecture that models the spatial and temporal structure in a hierarchical manner. Extensive experiments demonstrate that our method outperforms previous methods by a noticeable margin.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 31 of its 50 references that are in the map. First posted on arXiv 29 Jan 2024.

Builds on

31 of 50 references.

References · 31

  1. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  2. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  3. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 768
  4. 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 397
  5. 2019Learning joint reconstruction of hands and manipulated objectscited by 377
  6. 2022TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Textscited by 357
  7. 2021HuMoR: 3D Human Motion Model for Robust Pose Estimationcited by 319
  8. 2022ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulationcited by 306
  9. 2021Hand-Object Contact Consistency Reasoning for Human Grasps Generationcited by 160
  10. 2021GOAL: Generating 4D Whole-Body Motion for Hand-Object Graspingcited by 142
  11. 2020Leveraging Photometric Consistency over Time for Sparsely Supervised Hand-Object Reconstructioncited by 139
  12. 2021ContactOpt: Optimizing Contact to Improve Graspscited by 137
19 more
  1. 2020CPF: Learning a Contact Potential Field to Model the Hand-Object Interactioncited by 127
  2. 2023UniDexGrasp: Universal Robotic Dexterous Grasping via Learning Diverse Proposal Generation and Goal-Conditioned Policycited by 117
  3. 2020HOPE-Net: A Graph-based Model for Hand-Object Pose Estimationcited by 112
  4. 2021SAGA: Stochastic Whole-Body Grasping with Contactcited by 89
  5. 2019ContactGrasp: Functional Multi-finger Grasp Synthesis from Contactcited by 87
  6. 2022DexPoint: Generalizable Point Cloud Reinforcement Learning for Sim-to-Real Dexterous Manipulationcited by 78
  7. 2022TOCH: Spatio-Temporal Object-to-Hand Correspondence for Motion Refinementcited by 69
  8. 2022PSTNet: Point Spatio-Temporal Convolution on Point Cloud Sequencescited by 68
  9. 2021Towards unconstrained joint hand-object reconstruction from RGB videoscited by 57
  10. 2023CAMS: CAnonicalized Manipulation Spaces for Category-Level Functional Hand-Object Manipulation Synthesiscited by 54
  11. 2022Grasp'D: Differentiable Contact-rich Grasp Synthesis for Multi-fingered Handscited by 49
  12. 2020Joint Hand-object 3D Reconstruction from a Single Image with Cross-branch Feature Fusioncited by 39
  13. 2020Body2Hands: Learning to Infer 3D Hands from Conversational Gesture Body Dynamicscited by 34
  14. 2023Diverse Human Motion Prediction Guided by Multi-Level Spatial-Temporal Anchorscited by 34
  15. 2020Learning to dance: A graph convolutional adversarial network to generate realistic dance motions from audiocited by 33
  16. 2023PointCMP: Contrastive Mask Prediction for Self-supervised Learning on Point Cloud Videoscited by 22
  17. 2023Diverse 3D Hand Gesture Prediction from Body Dynamics by Bilateral Hand Disentanglementcited by 18
  18. 2022Tracking and Reconstructing Hand Object Interactions from Point Cloud Sequences in the Wildcited by 17
  19. 2022Spatial-Temporal Parallel Transformer for Arm-Hand Dynamic Estimationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 2; 2026: 1.

6 papers.

  1. 2024Large Motion Model for Unified Multi-Modal Motion Generationcited by 65
  2. 2024BimArt: A Unified Approach for the Synthesis of 3D Bimanual Interaction with Articulated Objectscited by 29
  3. 2025Diffusion-based 3D Hand Motion Recovery with Intuitive Physicscited by 5
  4. 2024CrowdMoGen: Zero-Shot Text-Driven Collective Motion Generationcited by 4
  5. 2025Decoupled Generative Modeling for Human-Object Interaction Synthesiscited by 2
  6. 2026DCGrasp: Distance-aware Controllable Grasp Generationcited by 0