mlsanityData to 14 Oct 2026

Teacher-Student Diffusion Model for Text-Driven 3D Hand Motion Generation

arXiv 2603.24407 ↗First posted 25 Mar 2026cs.CVcited by 0

On the map: Diffusion models → Diffusion and flow policies for robotics

also in 1 more
Abstract (arXiv)

Generating realistic 3D hand motion from natural language is vital for VR, robotics, and human-computer interaction. Existing methods either focus on full-body motion, overlooking detailed hand gestures, or require explicit 3D object meshes, limiting generality. We propose TSHaMo, a model-agnostic teacher-student diffusion framework for text-driven hand motion generation. The student model learns to synthesize motions from text alone, while the teacher leverages auxiliary signals (e.g., MANO parameters) to provide structured guidance during training. A co-training strategy enables the student to benefit from the teacher's intermediate predictions while remaining text-only at inference. Evaluated using two diffusion backbones on GRAB and H2O, TSHaMo consistently improves motion quality and diversity. Ablations confirm its robustness and flexibility in using diverse auxiliary inputs without requiring 3D objects at test time.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 17 of its 24 references that are in the map. First posted on arXiv 25 Mar 2026.

Builds on

17 of 24 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024DiffH2O: Diffusion-Based Synthesis of Hand-Object Interactions from Textual Descriptions26 Mar

Other references · 16

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2022Classifier-Free Diffusion Guidancecited by 5,847
  4. 2022Human Motion Diffusion Modelcited by 1,080
  5. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 780
  6. 2023MotionGPT: Human Motion as a Foreign Languagecited by 527
  7. 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 405
  8. 2021H2O: Two Hands Manipulating Objects for First Person Interaction Recognitioncited by 254
  9. 2022Embodied Hands: Modeling and Capturing Hands and Bodies Togethercited by 181
  10. 2021GOAL: Generating 4D Whole-Body Motion for Hand-Object Graspingcited by 144
  11. 2022IMos: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactionscited by 124
  12. 2023AttT2M: Text-Driven Human Motion Generation with Multi-Perspective Attention Mechanismcited by 117
4 more
  1. 2024Text2HOI: Text-guided 3D Motion Generation for Hand-Object Interactioncited by 86
  2. 2024StableMoFusion: Towards Robust and Efficient Diffusion-based Motion Generation Frameworkcited by 50
  3. 2024Unimotion: Unifying 3D Human Motion Synthesis and Understandingcited by 30
  4. 2022Effects of Auxiliary Knowledge on Continual Learningcited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Text-Driven 3D Hand Motion Generation from Sign Language Data
  2. 2026HandDreamer: Zero-Shot Text to 3D Hand Model Generation using Corrective Hand Shape Guidance
  3. 2023BOTH2Hands: Inferring 3D Hands from Both Text Prompts and Body Dynamics
  4. 2025SIGHT: Synthesizing Image-Text Conditioned and Geometry-Guided 3D Hand-Object Trajectories
  5. 2024NL2Contact: Natural Language Guided 3D Hand-Object Contact Modeling with Diffusion Model
  6. 2026MAD-HOI: Masked Autoregressive Diffusion for Generating Articulated Hand Object Interactions from Text
  7. 2025Bimanual 3D Hand Motion and Articulation Forecasting in Everyday Images
  8. 2023MACS: Mass Conditioned 3D Hand and Object Motion Synthesis