mlsanityData to 14 Oct 2026

HDMI: Learning Interactive Humanoid Whole-Body Control from Human Videos

arXiv 2509.16757 ↗First posted 20 Sep 2025cs.ROcited by 38

Not on the map. Why?

Abstract (arXiv)

Enabling robust whole-body humanoid-object interaction (HOI) remains challenging due to motion data scarcity and the contact-rich nature. We present HDMI (HumanoiD iMitation for Interaction), a simple and general framework that learns whole-body humanoid-object interaction skills directly from monocular RGB videos. Our pipeline (i) extracts and retargets human and object trajectories from unconstrained videos to build structured motion datasets, (ii) trains a reinforcement learning (RL) policy to co-track robot and object states with three key designs: a unified object representation, a residual action space, and a general interaction reward, and (iii) zero-shot deploys the RL policies on real humanoid robots. Extensive sim-to-real experiments on a Unitree G1 humanoid demonstrate the robustness and generality of our approach: HDMI achieves 67 consecutive door traversals and successfully performs 6 distinct loco-manipulation tasks in the real world and 14 tasks in simulation. Our results establish HDMI as a simple and general framework for acquiring interactive humanoid skills from human videos.

Key facts. Cited by 38 papers in the map (data through 14 Oct 2026). First posted on arXiv 20 Sep 2025.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 32.

38 papers.

Sort:

By area:

  1. 2025SPIDER: Scalable Physics-Informed Dexterous Retargetingcited by 32
  2. 2025TD-JEPA: Latent-predictive Representations for Zero-Shot Reinforcement Learningcited by 22
  3. 2025Opening the Sim-to-Real Door for Humanoid Pixel-to-Action Policy Transfercited by 15
  4. 2026HumanX: Toward Agile and Generalizable Humanoid Interaction Skills from Human Videoscited by 14
  5. 2026HEX: Humanoid-Aligned Experts for Cross-Embodiment Whole-Body Manipulationcited by 10
  6. 2025Humanoid Goalkeeper: Learning from Position Conditioned Task-Motion Constraintscited by 9
  7. 2026ULTRA: Unified Multimodal Control for Autonomous Humanoid Whole-Body Loco-Manipulationcited by 9
  8. 2025RoboMirror: Understand Before You Imitate for Video to Humanoid Locomotioncited by 7
  9. 2026Learning Whole-Body Human-Humanoid Interaction from Human-Human Demonstrationscited by 6
  10. 2026MeshMimic: Geometry-Aware Humanoid Motion Learning through 3D Scene Reconstructioncited by 6
  11. 2026SteadyTray: Learning Object Balancing Tasks in Humanoid Tray Transport via Residual Reinforcement Learningcited by 5
  12. 2026Spatial Memory for Out-of-Vision Manipulation in Vision-Language-Actioncited by 4
26 more
  1. 2026SceneBot: Contact-Prompted General Humanoid Whole Body Tracking with Scene-Interactioncited by 4
  2. 2026Learning Human-Like Badminton Skills for Humanoid Robotscited by 3
  3. 2026SUGAR: A Scalable Human-Video-Driven Generalizable Humanoid Loco-Manipulation Learning Frameworkcited by 3
  4. 2026SplitAdapter: Load-Aware Humanoid Loco-Manipulation via Factorized Adaptationcited by 3
  5. 2026REFINE-DP: Diffusion Policy Fine-tuning for Humanoid Loco-manipulation via Reinforcement Learningcited by 2
  6. 2026Imagine2Real: Towards Zero-shot Humanoid-Object Interaction via Video Generative Priorscited by 2
  7. 2026Scaling Behavior Foundation Model for Humanoid Robotscited by 2
  8. 2026AdaptManip: Learning Adaptive Whole-Body Object Lifting and Delivery with Online Recurrent State Estimationcited by 1
  9. 2026Generalizing from References using a Multi-Task Reference and Goal-Driven RL Frameworkcited by 1
  10. 2026EmbodMocap: In-the-Wild 4D Human-Scene Reconstruction for Embodied Agentscited by 1
  11. 2026DreamControl-v2: Simpler and Scalable Autonomous Humanoid Skills via Trainable Guided Diffusion Priorscited by 1
  12. 2026RoboNaldo: Accurate, Stable and Powerful Humanoid Soccer Shooting via Motion-Guided Curriculum Reinforcement Learningcited by 1
  13. 2026AnyBody: Free-Form Whole-Body Humanoid Control from Arbitrary Keypoint Guidancecited by 1
  14. 2026Weave: Learning Whole-Body Dexterous Loco-Manipulation from Human-Object Interactionscited by 1
  15. 2025CRISP: Contact-Guided Real2Sim from Monocular Video with Planar Scene Primitivescited by 0
  16. 2026LadderMan: Learning Humanoid Perceptive Ladder Climbingcited by 0
  17. 2026TaskNPoint: How to Teach Your Humanoid to Hit a Backhand in Minutescited by 0
  18. 2026LUCID: Latent-Skill Unified Control via Imagined Dynamics for Long-Horizon Humanoid Loco-Manipulationcited by 0
  19. 2026GigaBrain-WBC-0.5: A Behavior World Model for Robust Humanoid Whole-Body Tracking with Environment Interactioncited by 0
  20. 2026HIGenNTO: Scalable Humanoid Interaction Generation via Noise-Space Trajectory Optimizationcited by 0
  21. 2026Track-and-Complete: Learning Humanoid Skills from a Single Failed Human Videocited by 0
  22. 2026Counterfactual Video Generation Enables Scalable Humanoid Loco-Manipulationcited by 0
  23. 2026InterMimicGen: Scaling Humanoid Loco-Manipulation through Self-Evolving Motion Imitationcited by 0
  24. 2026Workhorse: Learning Robust Whole-Body Humanoid Loco-Manipulation from Human Datacited by 0
  25. 2026Being-M0.7: A Latent World-Action Model for Humanoid Robotscited by 0
  26. 2026From Video to Vision: Learning Generalizable Humanoid Carrying from Human Motioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Humanoid Manipulation Interface: Humanoid Whole-Body Manipulation from Robot-Free Demonstrations
  2. 2026HumanoidMimicGen: Data Generation for Loco-Manipulation via Whole-Body Planning
  3. 2026ZeroWBC: Learning Natural Whole-Body Humanoid Interaction from Human Egocentric Data
  4. 2024Learning Human-to-Humanoid Real-Time Whole-Body Teleoperation
  5. 2024Expressive Whole-Body Control for Humanoid Robots
  6. 2024Learning from Massive Human Videos for Universal Humanoid Pose Control
  7. 2026Towards a General Humanoid Loco-Manipulation Model via Egocentric Whole-Body Human Data Pretraining ↗
  8. 2026InterReal: A Unified Physics-Based Imitation Framework for Learning Human-Object Interaction Skills