mlsanityData to 14 Oct 2026

HandX: Scaling Bimanual Motion and Interaction Generation

arXiv 2603.28766 ↗First posted 30 Mar 2026cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Synthesizing human motion has advanced rapidly, yet realistic hand motion and bimanual interaction remain underexplored. Whole-body models often miss the fine-grained cues that drive dexterous behavior, finger articulation, contact timing, and inter-hand coordination, and existing resources lack high-fidelity bimanual sequences that capture nuanced finger dynamics and collaboration. To fill this gap, we present HandX, a unified foundation spanning data, annotation, and evaluation. We consolidate and filter existing datasets for quality, and collect a new motion-capture dataset targeting underrepresented bimanual interactions with detailed finger dynamics. For scalable annotation, we introduce a decoupled strategy that extracts representative motion features, e.g., contact events and finger flexion, and then leverages reasoning from large language models to produce fine-grained, semantically rich descriptions aligned with these features. Building on the resulting data and annotations, we benchmark diffusion and autoregressive models with versatile conditioning modes. Experiments demonstrate high-quality dexterous motion generation, supported by our newly proposed hand-focused metrics. We further observe clear scaling trends: larger models trained on larger, higher-quality datasets produce more semantically coherent bimanual motion. Our dataset is released to support future research.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 70 of its 87 references that are in the map. First posted on arXiv 30 Mar 2026.

Builds on

70 of 87 references.

References · 70

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  4. 2022Human Motion Diffusion Modelcited by 1,080
  5. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 780
  6. 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 588
  7. 2023MotionGPT: Human Motion as a Foreign Languagecited by 527
  8. 2022TEMOS: Generating diverse human motions from textual descriptionscited by 492
  9. 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 405
  10. 2023Human Motion Diffusion as a Generative Priorcited by 357
  11. 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 327
  12. 2022ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulationcited by 312
38 more
  1. 2023ReMoDiffuse: Retrieval-Augmented Motion Diffusion Modelcited by 287
  2. 2019Language2Pose: Natural Language Grounded Pose Forecastingcited by 257
  3. 2021H2O: Two Hands Manipulating Objects for First Person Interaction Recognitioncited by 254
  4. 2020InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Imagecited by 245
  5. 2023Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Datasetcited by 227
  6. 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 211
  7. 2023TMR: Text-to-Motion Retrieval Using Contrastive 3D Human Motion Synthesiscited by 198
  8. 2023InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusioncited by 182
  9. 2023HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real Worldcited by 166
  10. 2023Taming Diffusion Models for Audio-Driven Co-Speech Gesture Generationcited by 129
  11. 2024HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videoscited by 129
  12. 2022Joint Hand Motion and Interaction Hotspots Prediction from Egocentric Videoscited by 111
  13. 2017Text2Action: Generative Adversarial Synthesis from Language to Actioncited by 106
  14. 2023EMAGE: Towards Unified Holistic Co-Speech Gesture Generation via Expressive Masked Audio Gesture Modelingcited by 105
  15. 2023HumanTOMATO: Text-aligned Whole-body Motion Generationcited by 104
  16. 2024TACO: Benchmarking Generalizable Bimanual Tool-ACtion-Object Understandingcited by 88
  17. 2024Text2HOI: Text-guided 3D Motion Generation for Hand-Object Interactioncited by 86
  18. 2025MotionStreamer: Streaming Motion Generation via Diffusion-based Autoregressive Model in Causal Latent Spacecited by 86
  19. 2023DiffuseStyleGesture: Stylized Audio-Driven Co-Speech Gesture Generation with Diffusion Modelscited by 85
  20. 2023HOLD: Category-agnostic 3D Reconstruction of Interacting Hands and Objects from Videocited by 78
  21. 2022TOCH: Spatio-Temporal Object-to-Hand Correspondence for Motion Refinementcited by 70
  22. 2025Go to Zero: Towards Zero-shot Motion Generation with Million-scale Datacited by 68
  23. 2025HaWoR: World-Space Hand Motion Reconstruction from Egocentric Videoscited by 66
  24. 2024InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interactioncited by 64
  25. 2024DiffSHEG: A Diffusion-Based Approach for Real-Time Speech-driven Holistic 3D Expression and Gesture Generationcited by 62
  26. 2024Human4DiT: 360-degree Human Video Generation with 4D Diffusion Transformercited by 61
  27. 2024ParCo: Part-Coordinating Text-to-Motion Synthesiscited by 55
  28. 2024ScaMo: Exploring the Scaling Law in Autoregressive Motion Generation Modelcited by 53
  29. 2024Dyn-HaMR: Recovering 4D Interacting Hand Motion from a Dynamic Cameracited by 51
  30. 2023Neural Sign Actors: A diffusion model for 3D sign language production from textcited by 49
  31. 2025GestureLSM: Latent Shortcut based Co-Speech Gesture Generation with Spatial-Temporal Modelingcited by 44
  32. 2024GigaHands: A Massive Annotated Dataset of Bimanual Hand Activitiescited by 33
  33. 2025HOIGPT: Learning Long Sequence Hand-Object Interaction with Language Modelscited by 33
  34. 2025InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generationcited by 32
  35. 2024BimArt: A Unified Approach for the Synthesis of 3D Bimanual Interaction with Articulated Objectscited by 30
  36. 2023Stochastic Multi-Person 3D Motion Forecastingcited by 28
  37. 2023GRIP: Generating Interaction Poses Using Spatial Cues and Latent Consistencycited by 27
  38. 2023MotionScript: Natural Language Descriptions for Expressive 3D Human Motionscited by 21

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2024AsymDex: Asymmetry and Relative Coordinates for RL-based Bimanual Dexterity
  2. 2024ScrewMimic: Bimanual Imitation from Human Videos with Screw Space Projection
  3. 2026PatternDex: Learning Interaction Patterns to Guide Reinforcement Learning of Bimanual Dexterous Manipulation of Articulated Objects
  4. 2026Dex2HOI: Dexterous Bimanual Two-Object Interaction Generation
  5. 2025TOUCH: Text-guided Controllable Generation of Free-Form Hand-Object Interactions
  6. 2025ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning
  7. 2026Bench2Dex: Benchmarking Visuo-Tactile Bimanual Dexterous Manipulation Across Dexterous Hands
  8. 2025World Models for Learning Dexterous Hand-Object Interactions from Human Videos