mlsanityData to 14 Oct 2026

HUMOTO: A 4D Dataset of Mocap Human Object Interactions

arXiv 2504.10414 ↗First posted 14 Apr 2025cs.CVcited by 31

Not on the map. Why?

Abstract (arXiv)

We present Human Motions with Objects (HUMOTO), a high-fidelity dataset of human-object interactions for motion generation, computer vision, and robotics applications. Featuring 735 sequences (7,875 seconds at 30 fps), HUMOTO captures interactions with 63 precisely modeled objects and 72 articulated parts. Our innovations include a scene-driven LLM scripting pipeline creating complete, purposeful tasks with natural progression, and a mocap-and-camera recording setup to effectively handle occlusions. Spanning diverse activities from cooking to outdoor picnics, HUMOTO preserves both physical accuracy and logical task flow. Professional artists rigorously clean and verify each sequence, minimizing foot sliding and object penetrations. We also provide benchmarks compared to other datasets. HUMOTO's comprehensive full-body motion and simultaneous multi-object interactions address key data-capturing challenges and provide opportunities to advance realistic human-object interaction modeling across research domains with practical applications in animation, robotics, and embodied AI systems. Project: https://jiaxin-lu.github.io/humoto/ .

Key facts. Cited by 31 papers in the map (data through 14 Oct 2026). Builds on 60 of its 79 references that are in the map. First posted on arXiv 14 Apr 2025.

Builds on

60 of 79 references.

References · 60

  1. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  2. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
  3. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,162
  4. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 775
  5. 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 688
  6. 2018DensePose: Dense Human Pose Estimation In The Wildcited by 575
  7. 2023MotionGPT: Human Motion as a Foreign Languagecited by 527
  8. 2023FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objectscited by 429
  9. 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 405
  10. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 391
  11. 2023Humans in 4D: Reconstructing and Tracking Humans with Transformerscited by 350
  12. 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 327
38 more
  1. 2022ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulationcited by 312
  2. 2022Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activitiescited by 287
  3. 2022BEHAVE: Dataset and Method for Tracking Human Object Interactionscited by 249
  4. 2018Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objectscited by 241
  5. 2017First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotationscited by 230
  6. 2023Object Motion Guided Human Motion Synthesiscited by 212
  7. 2023InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusioncited by 182
  8. 2020Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wildcited by 152
  9. 2023CG-HOI: Contact-Guided 3D Human-Object Interaction Generationcited by 138
  10. 2022DexGraspNet: A Large-Scale Robotic Dexterous Grasp Dataset for General Objects Based on Simulationcited by 137
  11. 2024Scaling Up Dynamic Human-Scene Interaction Modelingcited by 137
  12. 2022COUCH: Towards Controllable Human-Chair Interactionscited by 135
  13. 2023SAM-6D: Segment Anything Model Meets Zero-Shot 6D Object Pose Estimationcited by 117
  14. 2023Affordance Diffusion: Synthesizing Hand-Object Interactionscited by 110
  15. 2023Controllable Human-Object Interaction Synthesiscited by 109
  16. 2023UniDexGrasp++: Improving Dexterous Grasping Policy Learning via Geometry-aware Curriculum and Iterative Generalist-Specialist Learningcited by 107
  17. 2024TRAM: Global Trajectory and Motion of 3D Humans from in-the-wild Videoscited by 106
  18. 2023Synthesizing Physical Character-Scene Interactionscited by 104
  19. 2022CHORE: Contact, Human and Object REconstruction from a single RGB imagecited by 95
  20. 2022InterCap: Joint Markerless 3D Tracking of Humans and Objects in Interactioncited by 93
  21. 2023HOI-Diff: Text-Driven Synthesis of 3D Human-Object Interactions using Diffusion Modelscited by 93
  22. 2024TACO: Benchmarking Generalizable Bimanual Tool-ACtion-Object Understandingcited by 88
  23. 2022Full-Body Articulated Human-Object Interactioncited by 78
  24. 2022NeuralDome: A Neural Modeling Pipeline on Multi-View Human-Object Interactionscited by 68
  25. 2022Approximate Convex Decomposition for 3D Meshes with Collision-Aware Concavity and Tree Searchcited by 67
  26. 2024InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interactioncited by 64
  27. 2024Human-Object Interaction from Human-Level Instructionscited by 64
  28. 2024ParaHome: Parameterizing Everyday Home Activities Towards 3D Generative Modeling of Human-Object Interactionscited by 57
  29. 2023Exploring Predicate Visual Context in Detecting Human-Object Interactionscited by 56
  30. 2024Autonomous Character-Scene Interaction Synthesis from Text Instructioncited by 56
  31. 2022Motion Policy Networkscited by 55
  32. 2024HOI-M3:Capture Multiple Humans and Objects Interaction within Contextual Environmentcited by 53
  33. 2023RLIPv2: Fast Scaling of Relational Language-Image Pre-trainingcited by 45
  34. 2023I'M HOI: Inertia-aware Monocular Capture of 3D Human-Object Interactionscited by 44
  35. 2024CooHOI: Learning Cooperative Human-Object Interaction with Manipulated Object Dynamicscited by 44
  36. 2023ViPLO: Vision Transformer based Pose-Conditioned Self-Loop Graph for Human-Object Interaction Detectioncited by 41
  37. 2023UGG: Unified Generative Graspingcited by 30
  38. 2024Joint Reconstruction of 3D Human and Object via Contact-Based Refinement Transformercited by 29

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 25.

31 papers.

Sort:

By area:

  1. 2025Embody 3D: A Large-scale Multimodal Motion and Behavior Datasetcited by 19
  2. 2025SpaceTimePilot: Generative Rendering of Dynamic Scenes Across Space and Timecited by 11
  3. 2026InterPrior: Scaling Generative Control for Physics-Based Human-Object Interactionscited by 11
  4. 2026VOID: Video Object and Interaction Deletioncited by 9
  5. 2026WHOLE: World-Grounded Hand-Object Lifted from Egocentric Videoscited by 8
  6. 2026FlowHOI: Flow-based Semantics-Grounded Generation of Hand-Object Interactions for Dexterous Robot Manipulationcited by 7
  7. 2025Advances in 4D Representation: Geometry, Motion, and Interactioncited by 4
  8. 2026SceneBot: Contact-Prompted General Humanoid Whole Body Tracking with Scene-Interactioncited by 4
  9. 2025Efficient and Scalable Monocular Human-Object Interaction Motion Reconstructioncited by 3
  10. 2026NymeriaPlus: Enriching Nymeria Dataset with Additional Annotations and Datacited by 3
  11. 2026InfBaGel: Human-Object-Scene Interaction Generation with Dynamic Perception and Iterative Refinementcited by 3
  12. 2026ACE-Data-0: Human-Centric Ambient Capture as Embodied Data Enginecited by 3
19 more
  1. 2026HSImul3R: Physics-in-the-Loop Reconstruction of Simulation-Ready Human-Scene Interactionscited by 2
  2. 2026Seeing Without Eyes: 4D Human-Scene Understanding from Wearable IMUscited by 2
  3. 2026RoMo: A Large-Scale, Richly Organized Dataset and Semantic Taxonomy for Human Motion Generationcited by 2
  4. 2026Dex2HOI: Dexterous Bimanual Two-Object Interaction Generationcited by 2
  5. 2026HOIMask: Towards Generative Masked Modeling for Human Object Interaction Generationcited by 2
  6. 2026EditStream: A Unified Autoregressive Framework for Interactive Video Generation and Editingcited by 2
  7. 2025Grounding Intelligence in Movementcited by 1
  8. 2026HiPHI: A Large-Scale Benchmark for High-Precision Human Motion and Object-Interactioncited by 1
  9. 2025Understanding the Representation of Older Adults in Motion Capture Locomotion Datasetscited by 0
  10. 2026DexterCap: An Affordable and Automated System for Capturing Dexterous Hand-Object Manipulationcited by 0
  11. 2026ArtHOI: Articulated Human-Object Interaction Synthesis by 4D Reconstruction from Video Priorscited by 0
  12. 2026OmniRobotHome: A Multi-Camera Home Platform for Real-Time Human-Robot Interactioncited by 0
  13. 2026Recovering Physically Plausible Human-Object Interactions from Monocular Videoscited by 0
  14. 2026IMU-HOI: A Symbiotic Framework for Coherent Human-Object Interaction and Motion Capture via Contact-Conscious Inertial Fusioncited by 0
  15. 2026RF-HOI: Recognize Human-Object Interaction with Radio Frequency Signalscited by 0
  16. 2026Efficient Human-Contact Representation for Human-Scene Interactioncited by 0
  17. 2026CL4D: Contrastive Language-4D Pretraining for Vision-Language Reasoning in Dynamic Scenescited by 0
  18. 2026Inter-X++: A Comprehensive Benchmark for Multimodal Human-Human Interaction Analysiscited by 0
  19. 2026COSMI: COmpositional Synthesis of Multi-object Interactionscited by 0