mlsanityData to 7 Oct 2026

HiPHI: A Large-Scale Benchmark for High-Precision Human Motion and Object-Interaction

arXiv 2608.16222 ↗First posted 17 Aug 2026cs.RO cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Humanoid intelligence requires learning over an extremely diverse space of whole-body motions and physically grounded interactions. However, existing embodied datasets remain fundamentally limited: internet-scale video data lack precise physical states and interaction grounding, while laboratory motion datasets provide high fidelity but only narrow behavioral coverage. This mismatch creates a critical bottleneck for scalable humanoid policy learning. We present HiPHI, a 600+ hour scale high-fidelity whole-body human motion dataset designed to systematically maximize coverage of the human motion and interaction manifold. HiPHI is theoretically guided by FrameNet, a linguistic framework organizing human primitives. Created using an optical motion capture pipeline, HiPHI provides sub-millimeter spatial marker tracking accuracy for full-body human motion and mesh-level object trajectories. We further introduce a benchmark suite evaluating motion-space diversity, interaction grounding, object consistency, and physical AI applications. Our analyses demonstrate that HiPHI significantly expands motion coverage compared to existing motion datasets while maintaining high-fidelity interaction quality, and establishes a scalable data foundation for training, evaluating, and generalizing humanoid policies in real-world embodied tasks, where similar extensions are also applicable to motion prior models in computer graphics. Project page: https://noitom-robotics.github.io/hiphi/

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 33 of its 45 references that are in the map. First posted on arXiv 17 Aug 2026.

Builds on

33 of 45 references.

References · 33

  1. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  2. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  3. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  4. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
  5. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
  6. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  7. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
  8. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
  9. 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 397
  10. 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 373
  11. 2016The KIT Motion-Language Datasetcited by 334
  12. 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 321
21 more
  1. 2021BABEL: Bodies, Action and Behavior with English Labelscited by 261
  2. 2022BEHAVE: Dataset and Method for Tracking Human Object Interactionscited by 246
  3. 2021Robust Motion In-betweeningcited by 223
  4. 2023Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Datasetcited by 220
  5. 2023Object Motion Guided Human Motion Synthesiscited by 206
  6. 2023Perpetual Humanoid Control for Real-time Simulated Avatarscited by 198
  7. 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 191
  8. 2024EgoMimic: Scaling Imitation Learning via Egocentric Videocited by 160
  9. 2025BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusioncited by 128
  10. 2022InterCap: Joint Markerless 3D Tracking of Humans and Objects in Interactioncited by 90
  11. 2025SONIC: Supersizing Motion Tracking for Natural Humanoid Whole-Body Controlcited by 83
  12. 2025OmniRetarget: Interaction-Preserving Data Generation for Humanoid Whole-Body Loco-Manipulation and Scene Interactioncited by 57
  13. 2023Neural Categorical Priors for Physics-Based Character Controlcited by 37
  14. 2024HIMO: A New Benchmark for Full-Body Human Interacting with Multiple Objectscited by 34
  15. 2025InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generationcited by 31
  16. 2023Lifelike Agility and Play in Quadrupedal Robots using Reinforcement Learning and Generative Pre-trained Modelscited by 30
  17. 2025Motion-X++: A Large-Scale Multimodal 3D Whole-body Human Motion Datasetcited by 30
  18. 2025HUMOTO: A 4D Dataset of Mocap Human Object Interactionscited by 30
  19. 2022MoCapAct: A Multi-Task Dataset for Simulated Humanoid Controlcited by 24
  20. 2025H2R: A Human-to-Robot Data Augmentation for Robot Pre-training from Videoscited by 21
  21. 2025EMMA: Scaling Mobile Manipulation via Egocentric Human Datacited by 20

Built on by

Not cited yet.