# HIMO: A New Benchmark for Full-Body Human Interacting with Multiple Objects

arXiv: https://arxiv.org/abs/2407.12371 · HTML: https://mlsanity.com/paper/2407.12371/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 65 of its 79 references that are in the map.
- First posted on arXiv 17 Jul 2024.

## Builds on (most cited first, 30 of 65)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Human Motion Diffusion Model](https://mlsanity.com/paper/2209.14916/) (2022)
- [T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representations](https://mlsanity.com/paper/2301.06052/) (2023)
- [Executing your Commands via Motion Diffusion in Latent Space](https://mlsanity.com/paper/2212.04048/) (2022)
- [MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation](https://mlsanity.com/paper/2302.08113/) (2023)
- [TEMOS: Generating diverse human motions from textual descriptions](https://mlsanity.com/paper/2204.14109/) (2022)
- [GRAB: A Dataset of Whole-Body Human Grasping of Objects](https://mlsanity.com/paper/2008.11200/) (2020)
- [Human Motion Diffusion as a Generative Prior](https://mlsanity.com/paper/2303.01418/) (2023)
- [The KIT Motion-Language Dataset](https://mlsanity.com/paper/1607.03827/) (2016)
- [HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interaction](https://mlsanity.com/paper/2203.01577/) (2022)
- [ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation](https://mlsanity.com/paper/2204.13662/) (2022)
- [BABEL: Bodies, Action and Behavior with English Labels](https://mlsanity.com/paper/2106.09696/) (2021)
- [BEHAVE: Dataset and Method for Tracking Human Object Interactions](https://mlsanity.com/paper/2204.06950/) (2022)
- [Guided Motion Diffusion for Controllable Human Motion Synthesis](https://mlsanity.com/paper/2305.12577/) (2023)
- [MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesis](https://mlsanity.com/paper/2212.04495/) (2022)
- [Object Motion Guided Human Motion Synthesis](https://mlsanity.com/paper/2309.16237/) (2023)
- [InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactions](https://mlsanity.com/paper/2304.05684/) (2023)
- [ContactPose: A Dataset of Grasps with Object Contact and Hand Pose](https://mlsanity.com/paper/2007.09545/) (2020)
- [InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusion](https://mlsanity.com/paper/2308.16905/) (2023)
- [TEACH: Temporal Action Composition for 3D Humans](https://mlsanity.com/paper/2209.04066/) (2022)
- [HUMANISE: Language-conditioned Human Motion Generation in 3D Scenes](https://mlsanity.com/paper/2210.09729/) (2022)
- [PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Images](https://mlsanity.com/paper/2207.06400/) (2022)
- [ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imaging](https://mlsanity.com/paper/1904.06830/) (2019)
- [GOAL: Generating 4D Whole-Body Motion for Hand-Object Grasping](https://mlsanity.com/paper/2112.11454/) (2021)

## Built on by (most cited first, 30 of 36)

- [InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactions](https://mlsanity.com/paper/2502.20390/) (2025)
- [GigaHands: A Massive Annotated Dataset of Bimanual Hand Activities](https://mlsanity.com/paper/2412.04244/) (2024)
- [InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generation](https://mlsanity.com/paper/2509.09555/) (2025)
- [A Survey on Human Interaction Motion Generation](https://mlsanity.com/paper/2503.12763/) (2025)
- [CoDA: Coordinated Diffusion Noise Optimization for Whole-Body Manipulation of Articulated Objects](https://mlsanity.com/paper/2505.21437/) (2025)
- [InterPrior: Scaling Generative Control for Physics-Based Human-Object Interactions](https://mlsanity.com/paper/2602.06035/) (2026)
- [HOI-Dyn: Learning Interaction Dynamics for Human-Object Motion Diffusion](https://mlsanity.com/paper/2507.01737/) (2025)
- [Perceiving and Acting in First-Person: A Dataset and Benchmark for Egocentric Human-Object-Human Interactions](https://mlsanity.com/paper/2508.04681/) (2025)
- [3D Human Interaction Generation: A Survey](https://mlsanity.com/paper/2503.13120/) (2025)
- [GenHOI: Generalizing Text-driven 4D Human-Object Interaction Synthesis for Unseen Objects](https://mlsanity.com/paper/2506.15483/) (2025)
- [EgoGrasp: World-Space Hand-Object Interaction Estimation from Egocentric Videos](https://mlsanity.com/paper/2601.01050/) (2026)
- [GenM3: Generative Pretrained Multi-path Motion Model for Text Conditional Human Motion Generation](https://mlsanity.com/paper/2503.14919/) (2025)
- [HOI-PAGE: Zero-Shot Human-Object Interaction Generation with Part Affordance Guidance](https://mlsanity.com/paper/2506.07209/) (2025)
- [Text-Driven 3D Hand Motion Generation from Sign Language Data](https://mlsanity.com/paper/2508.15902/) (2025)
- [PA-HOI: A Physics-Aware Human and Object Interaction Dataset](https://mlsanity.com/paper/2508.06205/) (2025)
- [InterPose: Learning to Generate Human-Object Interactions from Large-Scale Web Videos](https://mlsanity.com/paper/2509.00767/) (2025)
- [OnlineHOI: Towards Online Human-Object Interaction Generation and Perception](https://mlsanity.com/paper/2509.12250/) (2025)
- [MMHOI: Modeling Complex 3D Multi-Human Multi-Object Interactions](https://mlsanity.com/paper/2510.07828/) (2025)
- [InfBaGel: Human-Object-Scene Interaction Generation with Dynamic Perception and Iterative Refinement](https://mlsanity.com/paper/2604.04843/) (2026)
- [ACE-Data-0: Human-Centric Ambient Capture as Embodied Data Engine](https://mlsanity.com/paper/2607.28625/) (2026)
- [Affordance-Guided Diffusion Prior for 3D Hand Reconstruction](https://mlsanity.com/paper/2510.00506/) (2025)
- [Multimodal Priors-Augmented Text-Driven 3D Human-Object Interaction Generation](https://mlsanity.com/paper/2602.10659/) (2026)
- [Dex2HOI: Dexterous Bimanual Two-Object Interaction Generation](https://mlsanity.com/paper/2605.30444/) (2026)
- [Surface Keypoint Representation for Multi-Object and Articulated Human-Object Interaction Generation](https://mlsanity.com/paper/2608.03158/) (2026)
- [EigenActor: Variant Body-Object Interaction Generation Evolved from Invariant Action Basis Reasoning](https://mlsanity.com/paper/2503.00382/) (2025)
- [Prime and Reach: Synthesising Body Motion for Gaze-Primed Object Reach](https://mlsanity.com/paper/2512.16456/) (2025)
- [InterPhys: Physics-aware Human Motion Synthesis in a Dynamic Scene](https://mlsanity.com/paper/2605.01036/) (2026)
- [HiPHI: A Large-Scale Benchmark for High-Precision Human Motion and Object-Interaction](https://mlsanity.com/paper/2608.16222/) (2026)
- [MoGIC: Boosting Motion Generation via Intention Understanding and Visual Context](https://mlsanity.com/paper/2510.02722/) (2025)
- [DexterCap: An Affordable and Automated System for Capturing Dexterous Hand-Object Manipulation](https://mlsanity.com/paper/2601.05844/) (2026)

## Abstract (arXiv)

Generating human-object interactions (HOIs) is critical with the tremendous advances of digital avatars. Existing datasets are typically limited to humans interacting with a single object while neglecting the ubiquitous manipulation of multiple objects. Thus, we propose HIMO, a large-scale MoCap dataset of full-body human interacting with multiple objects, containing 3.3K 4D HOI sequences and 4.08M 3D HOI frames. We also annotate HIMO with detailed textual descriptions and temporal segments, benchmarking two novel tasks of HOI synthesis conditioned on either the whole text prompt or the segmented text prompts as fine-grained timeline control. To address these novel tasks, we propose a dual-branch conditional diffusion model with a mutual interaction module for HOI synthesis. Besides, an auto-regressive generation pipeline is also designed to obtain smooth transitions between HOI segments. Experimental results demonstrate the generalization ability to unseen object geometries and temporal compositions.