# Weave: Learning Whole-Body Dexterous Loco-Manipulation from Human-Object Interactions

arXiv: https://arxiv.org/abs/2609.16683 · HTML: https://mlsanity.com/paper/2609.16683/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 54 references that are in the map.
- First posted on arXiv 15 Sep 2026.

## Builds on (most cited first, 30 of 49)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024)
- [π0.5: a Vision-Language-Action Model with Open-World Generalization](https://mlsanity.com/paper/2504.16054/) (2025)
- [GR00T N1: An Open Foundation Model for Generalist Humanoid Robots](https://mlsanity.com/paper/2503.14734/) (2025)
- [Muon is Scalable for LLM Training](https://mlsanity.com/paper/2502.16982/) (2025)
- [Perpetual Humanoid Control for Real-time Simulated Avatars](https://mlsanity.com/paper/2305.06456/) (2023)
- [Object Motion Guided Human Motion Synthesis](https://mlsanity.com/paper/2309.16237/) (2023)
- [OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning](https://mlsanity.com/paper/2406.08858/) (2024)
- [BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusion](https://mlsanity.com/paper/2508.08241/) (2025)
- [SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learning](https://mlsanity.com/paper/2410.09754/) (2024)
- [Retargeting Matters: General Motion Retargeting for Humanoid Motion Tracking](https://mlsanity.com/paper/2510.02252/) (2025)
- [InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactions](https://mlsanity.com/paper/2502.20390/) (2025)
- [OmniRetarget: Interaction-Preserving Data Generation for Humanoid Whole-Body Loco-Manipulation and Scene Interaction](https://mlsanity.com/paper/2509.26633/) (2025)
- [Human-Object Interaction from Human-Level Instructions](https://mlsanity.com/paper/2406.17840/) (2024)
- [Omnigrasp: Grasping Diverse Objects with Simulated Humanoids](https://mlsanity.com/paper/2407.11385/) (2024)
- [ResMimic: From General Motion Tracking to Humanoid Whole-body Loco-Manipulation via Residual Learning](https://mlsanity.com/paper/2510.05070/) (2025)
- [Opt2Skill: Imitating Dynamically-feasible Whole-Body Trajectories for Versatile Humanoid Loco-Manipulation](https://mlsanity.com/paper/2409.20514/) (2024)
- [HDMI: Learning Interactive Humanoid Whole-Body Control from Human Videos](https://mlsanity.com/paper/2509.16757/) (2025)
- [Kimodo: Scaling Controllable Human Motion Generation](https://mlsanity.com/paper/2603.15546/) (2026)
- [WholeBodyVLA: Towards Unified Latent VLA for Whole-Body Loco-Manipulation Control](https://mlsanity.com/paper/2512.11047/) (2025)
- [Psi0: An Open Foundation Model Towards Universal Humanoid Loco-Manipulation](https://mlsanity.com/paper/2603.12263/) (2026)
- [VisualMimic: Visual Humanoid Loco-Manipulation via Motion Tracking and Generation](https://mlsanity.com/paper/2509.20322/) (2025)
- [VIRAL: Visual Sim-to-Real at Scale for Humanoid Loco-Manipulation](https://mlsanity.com/paper/2511.15200/) (2025)
- [Locate 3D: Real-World Object Localization via Self-Supervised Learning in 3D](https://mlsanity.com/paper/2504.14151/) (2025)
- [PhysHSI: Towards a Real-World Generalizable and Natural Humanoid-Scene Interaction System](https://mlsanity.com/paper/2510.11072/) (2025)
- [Hand-Eye Autonomous Delivery: Learning Humanoid Navigation, Locomotion and Reaching](https://mlsanity.com/paper/2508.03068/) (2025)
- [InterPrior: Scaling Generative Control for Physics-Based Human-Object Interactions](https://mlsanity.com/paper/2602.06035/) (2026)
- [HEX: Humanoid-Aligned Experts for Cross-Embodiment Whole-Body Manipulation](https://mlsanity.com/paper/2604.07993/) (2026)
- [ULTRA: Unified Multimodal Control for Autonomous Humanoid Whole-Body Loco-Manipulation](https://mlsanity.com/paper/2603.03279/) (2026)

## Built on by (most cited first, 1 of 1)

- [InterMimicGen: Scaling Humanoid Loco-Manipulation through Self-Evolving Motion Imitation](https://mlsanity.com/paper/2610.06850/) (2026)

## Abstract (arXiv)

Learning humanoid-object interaction requires coordinating whole-body balance, locomotion, and dexterous hand contact to control both robot and object motion. Human demonstrations provide examples of coordinated interaction, but transferring these behaviors to humanoid robots requires learning how to establish and maintain effective contacts under different embodiments and dynamics. We present Weave, a unified framework for learning whole-body dexterous humanoid-object interaction from captured human demonstrations. Weave first converts captured human-object interactions into executable robot-object references through contact-aware retargeting and approach-motion completion. At its core is a contact- and geometry-aware policy that jointly commands 29 body joints and 12 actuated finger joints across multiple objects and interaction sequences. Evaluation across nine objects yields a 92.5% success rate on trained interactions and, without any additional training, 65.0% on sequences never seen during training. We additionally release ~9,000 physically executed rollouts spanning ~23 hours, providing robot-object trajectories with contact annotations for downstream interaction-policy learning and physically consistent HOI motion generation. Project website: https://xiaohu-art.github.io/Weave/