JointHOI: Jointly Generating Contact Maps Enhances Hand Object Interaction Generation
Not on the map. Why?
Abstract (arXiv)
Text driven hand object interaction (HOI) generation is gaining attention for immersive applications and robotics, yet producing physically plausible interactions remains challenging. Even when individual motions appear natural, small contact errors can cause conspicuous artifacts such as floating and interpenetration. Prior methods mitigate these issues using explicit contact cues or implicit grasp priors, but typically rely on multi stage pipelines and fail to model temporally evolving contact. We present JointHOI, a single stage diffusion framework that jointly generates 3D hand object motion and dynamic, distance based contact maps from text. By treating contact as an auxiliary inner modality, joint generation enables the model to learn contact motion coupling during training. At inference, contact guided sampling enforces consistency between generated contact maps and motion implied geometry, improving temporal stability and reducing penetration and floating. Experiments on GRAB and ARCTIC demonstrate consistent improvements in text adherence and physical plausibility over prior methods.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 32 of its 38 references that are in the map. First posted on arXiv 2 Jul 2026.
Builds on
32 of 38 references.
References · 32
- 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
- 2022Human Motion Diffusion Modelcited by 1,062
- 2022Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Taskscited by 420
- 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 397
- 2022ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulationcited by 306
- 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 301
- 2023Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuningcited by 168
- 2021Hand-Object Contact Consistency Reasoning for Human Grasps Generationcited by 160
- 2020Reconstructing Hand-Object Interactions in the Wildcited by 143
- 2021ContactOpt: Optimizing Contact to Improve Graspscited by 137
- 2023CG-HOI: Contact-Guided 3D Human-Object Interaction Generationcited by 137
- 2022IMos: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactionscited by 121
20 more
- 20234M: Massively Multimodal Masked Modelingcited by 118
- 2025VideoJAM: Joint Appearance-Motion Representations for Enhanced Motion Generation in Video Modelscited by 112
- 2023Optimizing Diffusion Noise Can Serve As Universal Motion Priorscited by 97
- 2024Text2HOI: Text-guided 3D Motion Generation for Hand-Object Interactioncited by 84
- 2024Move as You Say, Interact as You Can: Language-guided Human Motion Generation with Scene Affordancecited by 79
- 2024GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Controlcited by 76
- 2024World-consistent Video Diffusion with Explicit 3D Modelingcited by 62
- 2024DiffH2O: Diffusion-Based Synthesis of Hand-Object Interactions from Textual Descriptionscited by 60
- 2023ContactGen: Generative Contact Modeling for Grasp Generationcited by 52
- 20244M-21: An Any-to-Any Vision Model for Tens of Tasks and Modalitiescited by 52
- 2025Boosting Generative Image Modeling via Joint Image-Feature Synthesiscited by 42
- 2025HOIGPT: Learning Long Sequence Hand-Object Interaction with Language Modelscited by 32
- 2024BimArt: A Unified Approach for the Synthesis of 3D Bimanual Interaction with Articulated Objectscited by 29
- 2025Guiding Human-Object Interactions with Rich Geometry and Relationscited by 24
- 2025Orchid: Image Latent Diffusion for Joint Appearance and Geometry Generationcited by 17
- 2025HOIDiNi: Human-Object Interaction through Diffusion Noise Optimizationcited by 16
- 2025JointDiT: Enhancing RGB-Depth Joint Modeling with Diffusion Transformerscited by 11
- 2024NL2Contact: Natural Language Guided 3D Hand-Object Contact Modeling with Diffusion Modelcited by 10
- 2025CoDA: Coordinated Diffusion Noise Optimization for Whole-Body Manipulation of Articulated Objectscited by 10
- 2025JAM-Flow: Joint Audio-Motion Synthesis with Flow Matchingcited by 3
Built on by
Citing papers per year: 2026: 2.
2 papers.