mlsanityData to 7 Oct 2026

HIMO: A New Benchmark for Full-Body Human Interacting with Multiple Objects

arXiv 2407.12371 ↗First posted 17 Jul 2024cs.CV cs.AIcited by 34

Not on the map. Why?

Abstract (arXiv)

Generating human-object interactions (HOIs) is critical with the tremendous advances of digital avatars. Existing datasets are typically limited to humans interacting with a single object while neglecting the ubiquitous manipulation of multiple objects. Thus, we propose HIMO, a large-scale MoCap dataset of full-body human interacting with multiple objects, containing 3.3K 4D HOI sequences and 4.08M 3D HOI frames. We also annotate HIMO with detailed textual descriptions and temporal segments, benchmarking two novel tasks of HOI synthesis conditioned on either the whole text prompt or the segmented text prompts as fine-grained timeline control. To address these novel tasks, we propose a dual-branch conditional diffusion model with a mutual interaction module for HOI synthesis. Besides, an auto-regressive generation pipeline is also designed to obtain smooth transitions between HOI segments. Experimental results demonstrate the generalization ability to unseen object geometries and temporal compositions.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Builds on 65 of its 79 references that are in the map. First posted on arXiv 17 Jul 2024.

Builds on

65 of 79 references.

References · 65

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2017Neural Discrete Representation Learningcited by 4,650
  5. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  6. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  7. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
  8. 2022Human Motion Diffusion Modelcited by 1,062
  9. 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 577
  10. 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 567
  11. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 529
  12. 2022TEMOS: Generating diverse human motions from textual descriptionscited by 486
38 more
  1. 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 397
  2. 2023Human Motion Diffusion as a Generative Priorcited by 354
  3. 2016The KIT Motion-Language Datasetcited by 334
  4. 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 321
  5. 2022ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulationcited by 306
  6. 2021BABEL: Bodies, Action and Behavior with English Labelscited by 261
  7. 2022BEHAVE: Dataset and Method for Tracking Human Object Interactionscited by 246
  8. 2023Guided Motion Diffusion for Controllable Human Motion Synthesiscited by 221
  9. 2022MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesiscited by 218
  10. 2023Object Motion Guided Human Motion Synthesiscited by 206
  11. 2023InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactionscited by 205
  12. 2023InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusioncited by 181
  13. 2020ContactPose: A Dataset of Grasps with Object Contact and Hand Posecited by 180
  14. 2022TEACH: Temporal Action Composition for 3D Humanscited by 173
  15. 2022HUMANISE: Language-conditioned Human Motion Generation in 3D Scenescited by 156
  16. 2022PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Imagescited by 152
  17. 2019ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imagingcited by 145
  18. 2021GOAL: Generating 4D Whole-Body Motion for Hand-Object Graspingcited by 142
  19. 2023TriDet: Temporal Action Detection with Relative Boundary Modelingcited by 141
  20. 2021Collaborative Regression of Expressive Bodies using Moderationcited by 137
  21. 2023CG-HOI: Contact-Guided 3D Human-Object Interaction Generationcited by 137
  22. 2020Coherent Reconstruction of Multiple Humans from a Single Imagecited by 136
  23. 2022COUCH: Towards Controllable Human-Chair Interactionscited by 134
  24. 2023One-Stage 3D Whole-Body Mesh Recovery with Component Aware Transformercited by 128
  25. 2021Keypoint Transformer: Solving Joint Identification in Challenging Hands and Object Interactions for Accurate 3D Pose Estimationcited by 122
  26. 2022IMos: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactionscited by 121
  27. 2021D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactionscited by 109
  28. 2023Controllable Human-Object Interaction Synthesiscited by 107
  29. 2019Efficient Learning on Point Clouds with Basis Point Setscited by 101
  30. 2023DiffCollage: Parallel Generation of Large Content with Diffusion Modelscited by 100
  31. 2023Inter-X: Towards Versatile Human-Human Interaction Analysiscited by 98
  32. 2022InterCap: Joint Markerless 3D Tracking of Humans and Objects in Interactioncited by 90
  33. 2021SAGA: Stochastic Whole-Body Grasping with Contactcited by 89
  34. 2023HOI-Diff: Text-Driven Synthesis of 3D Human-Object Interactions using Diffusion Modelscited by 89
  35. 2024Multi-Track Timeline Control for Text-Driven 3D Human Motion Generationcited by 88
  36. 2022Temporal Action Segmentation: An Analysis of Modern Techniquescited by 85
  37. 2024TACO: Benchmarking Generalizable Bimanual Tool-ACtion-Object Understandingcited by 85
  38. 2023NIFTY: Neural Object Interaction Fields for Guided Human Motion Synthesiscited by 78

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 19; 2026: 14.

34 papers.

Sort:

By area:

  1. 2025InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactionscited by 67
  2. 2024GigaHands: A Massive Annotated Dataset of Bimanual Hand Activitiescited by 32
  3. 2025InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generationcited by 31
  4. 2025A Survey on Human Interaction Motion Generationcited by 25
  5. 2025CoDA: Coordinated Diffusion Noise Optimization for Whole-Body Manipulation of Articulated Objectscited by 10
  6. 2025HOI-Dyn: Learning Interaction Dynamics for Human-Object Motion Diffusioncited by 10
  7. 2026InterPrior: Scaling Generative Control for Physics-Based Human-Object Interactionscited by 10
  8. 2025GenHOI: Generalizing Text-driven 4D Human-Object Interaction Synthesis for Unseen Objectscited by 7
  9. 2025Perceiving and Acting in First-Person: A Dataset and Benchmark for Egocentric Human-Object-Human Interactionscited by 7
  10. 2026EgoGrasp: World-Space Hand-Object Interaction Estimation from Egocentric Videoscited by 7
  11. 20253D Human Interaction Generation: A Surveycited by 6
  12. 2025GenM3: Generative Pretrained Multi-path Motion Model for Text Conditional Human Motion Generationcited by 5
22 more
  1. 2025Text-Driven 3D Hand Motion Generation from Sign Language Datacited by 5
  2. 2025HOI-PAGE: Zero-Shot Human-Object Interaction Generation with Part Affordance Guidancecited by 4
  3. 2025PA-HOI: A Physics-Aware Human and Object Interaction Datasetcited by 3
  4. 2026InfBaGel: Human-Object-Scene Interaction Generation with Dynamic Perception and Iterative Refinementcited by 3
  5. 2025OnlineHOI: Towards Online Human-Object Interaction Generation and Perceptioncited by 2
  6. 2025Affordance-Guided Diffusion Prior for 3D Hand Reconstructioncited by 2
  7. 2025MMHOI: Modeling Complex 3D Multi-Human Multi-Object Interactionscited by 2
  8. 2026Multimodal Priors-Augmented Text-Driven 3D Human-Object Interaction Generationcited by 2
  9. 2025EigenActor: Variant Body-Object Interaction Generation Evolved from Invariant Action Basis Reasoningcited by 1
  10. 2025InterPose: Learning to Generate Human-Object Interactions from Large-Scale Web Videoscited by 1
  11. 2025Prime and Reach: Synthesising Body Motion for Gaze-Primed Object Reachcited by 1
  12. 2026InterPhys: Physics-aware Human Motion Synthesis in a Dynamic Scenecited by 1
  13. 2026Dex2HOI: Dexterous Bimanual Two-Object Interaction Generationcited by 1
  14. 2026Surface Keypoint Representation for Multi-Object and Articulated Human-Object Interaction Generationcited by 1
  15. 2025MoGIC: Boosting Motion Generation via Intention Understanding and Visual Contextcited by 0
  16. 2026DexterCap: An Affordable and Automated System for Capturing Dexterous Hand-Object Manipulationcited by 0
  17. 2026Rendering Multi-Human and Multi-Object with 3D Gaussian Splattingcited by 0
  18. 2026Dynamic Full-body Motion Agent with Object Interaction via Blending Pre-trained Modular Controllerscited by 0
  19. 2026IMAGIN-4D: Image-Guided Controllable Interaction Generationcited by 0
  20. 2026ACE-Data-0: Human-Centric Ambient Capture as Embodied Data Enginecited by 0
  21. 2026HiPHI: A Large-Scale Benchmark for High-Precision Human Motion and Object-Interactioncited by 0
  22. 2026PAMI: Part Anchored Motion for Text to Human-Object Interaction Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024OOD-HOI: Text-Driven 3D Whole-Body Human-Object Interactions Generation Beyond Training Domains
  2. 2024AvatarGO: Zero-shot 4D Human-Object Interaction Generation and Animation
  3. 2025Learning to Generate Human-Human-Object Interactions from Textual Descriptions
  4. 2026COSMI: COmpositional Synthesis of Multi-object Interactions
  5. 2026OneHOI: Unifying Human-Object Interaction Generation and Editing
  6. 2025Efficient Explicit Joint-level Interaction Modeling with Mamba for Text-guided HOI Generation
  7. 2025InteractAnything: Zero-shot Human Object Interaction Synthesis via LLM Feedback and Object Affordance Parsing
  8. 2025HOIDiNi: Human-Object Interaction through Diffusion Noise Optimization