HiPHI: A Large-Scale Benchmark for High-Precision Human Motion and Object-Interaction
Not on the map. Why?
Abstract (arXiv)
Humanoid intelligence requires learning over an extremely diverse space of whole-body motions and physically grounded interactions. However, existing embodied datasets remain fundamentally limited: internet-scale video data lack precise physical states and interaction grounding, while laboratory motion datasets provide high fidelity but only narrow behavioral coverage. This mismatch creates a critical bottleneck for scalable humanoid policy learning. We present HiPHI, a 600+ hour scale high-fidelity whole-body human motion dataset designed to systematically maximize coverage of the human motion and interaction manifold. HiPHI is theoretically guided by FrameNet, a linguistic framework organizing human primitives. Created using an optical motion capture pipeline, HiPHI provides sub-millimeter spatial marker tracking accuracy for full-body human motion and mesh-level object trajectories. We further introduce a benchmark suite evaluating motion-space diversity, interaction grounding, object consistency, and physical AI applications. Our analyses demonstrate that HiPHI significantly expands motion coverage compared to existing motion datasets while maintaining high-fidelity interaction quality, and establishes a scalable data foundation for training, evaluating, and generalizing humanoid policies in real-world embodied tasks, where similar extensions are also applicable to motion prior models in computer graphics. Project page: https://noitom-robotics.github.io/hiphi/
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 33 of its 45 references that are in the map. First posted on arXiv 17 Aug 2026.
Builds on
33 of 45 references.
References · 33
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
- 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
- 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
- 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
- 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
- 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
- 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
- 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
- 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 397
- 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 373
- 2016The KIT Motion-Language Datasetcited by 334
- 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 321
21 more
- 2021BABEL: Bodies, Action and Behavior with English Labelscited by 261
- 2022BEHAVE: Dataset and Method for Tracking Human Object Interactionscited by 246
- 2021Robust Motion In-betweeningcited by 223
- 2023Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Datasetcited by 220
- 2023Object Motion Guided Human Motion Synthesiscited by 206
- 2023Perpetual Humanoid Control for Real-time Simulated Avatarscited by 198
- 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 191
- 2024EgoMimic: Scaling Imitation Learning via Egocentric Videocited by 160
- 2025BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusioncited by 128
- 2022InterCap: Joint Markerless 3D Tracking of Humans and Objects in Interactioncited by 90
- 2025SONIC: Supersizing Motion Tracking for Natural Humanoid Whole-Body Controlcited by 83
- 2025OmniRetarget: Interaction-Preserving Data Generation for Humanoid Whole-Body Loco-Manipulation and Scene Interactioncited by 57
- 2023Neural Categorical Priors for Physics-Based Character Controlcited by 37
- 2024HIMO: A New Benchmark for Full-Body Human Interacting with Multiple Objectscited by 34
- 2025InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generationcited by 31
- 2023Lifelike Agility and Play in Quadrupedal Robots using Reinforcement Learning and Generative Pre-trained Modelscited by 30
- 2025Motion-X++: A Large-Scale Multimodal 3D Whole-body Human Motion Datasetcited by 30
- 2025HUMOTO: A 4D Dataset of Mocap Human Object Interactionscited by 30
- 2022MoCapAct: A Multi-Task Dataset for Simulated Humanoid Controlcited by 24
- 2025H2R: A Human-to-Robot Data Augmentation for Robot Pre-training from Videoscited by 21
- 2025EMMA: Scaling Mobile Manipulation via Egocentric Human Datacited by 20
Built on by
Not cited yet.