OpenHLM: An Empirical Recipe for Whole-Body Humanoid Loco-Manipulation
On the map: Vision-language-action models → Tactile, force-aware and humanoid VLA
Abstract (arXiv)
Whole-body humanoid loco-manipulation requires coordinating the robot's entire kinematic chain. However, most existing systems typically decouple the upper and lower bodies into separate controllers, limiting such coordination and yielding behaviors similar to those of a wheeled dual-arm platform. In this paper, we ask what it takes to build a whole-body native vision-language-action (VLA) model that maps language and pixels directly to all of the humanoid's degrees of freedom. We conduct a systematic empirical study organized as a roadmap of one-variable-at-a-time experiments across three phases: whole-body teleoperation, VLA model design, and heterogeneous co-training. Our study yields several intriguing findings: a joint-based whole-body teleoperation interface outperforms alternatives that only partially expose the humanoid's degrees of freedom; a VLA pretrained on static and wheeled dual-arm platforms transfers surprisingly well to a humanoid's full action space; and co-training with HuMI, the humanoid analog of UMI, extends the policy to new objects and instructions without additional whole-body teleoperation on those targets. Following this roadmap yields OpenHLM, an open-source recipe for whole-body humanoid loco-manipulation. In a challenging long-horizon task that spans a wide vertical range of the humanoid, OpenHLM outperforms two state-of-the-art humanoid VLA baselines (GR00T N1.6 and Psi0) using less than half the total demonstration time. Our code, training data, and model checkpoints are available at [https://openhlm-project.github.io/].
Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Tactile, force-aware and humanoid VLA, one of 13 directions in Vision-language-action models. Builds on 44 of its 54 references that are in the map. First posted on arXiv 20 Jun 2026.
Builds on
44 of 54 references.
References · 44
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
- 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
- 2024PaliGemma: A versatile 3B VLM for transfercited by 679
- 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
- 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 477
32 more
- 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 386
- 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 384
- 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 373
- 2024Latent Action Pretraining from Videoscited by 302
- 2025π*0.6: a VLA That Learns From Experiencecited by 280
- 2023Project Aria: A New Tool for Egocentric Multi-Modal AI Researchcited by 201
- 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 191
- 2024OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learningcited by 167
- 2024HumanPlus: Humanoid Shadowing and Imitation from Humanscited by 164
- 2024Data Scaling Laws in Imitation Learning for Robotic Manipulationcited by 144
- 2025BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusioncited by 128
- 2026π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilitiescited by 127
- 2024Expressive Whole-Body Control for Humanoid Robotscited by 117
- 2025EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videoscited by 116
- 2026Generative Modeling via Driftingcited by 98
- 2025TWIST: Teleoperated Whole-Body Imitation Systemcited by 86
- 2025SONIC: Supersizing Motion Tracking for Natural Humanoid Whole-Body Controlcited by 83
- 2025GMT: General Motion Tracking for Humanoid Whole-Body Controlcited by 72
- 2025HOMIE: Humanoid Loco-Manipulation with Isomorphic Exoskeleton Cockpitcited by 65
- 2025Humanoid Policy ~ Human Policycited by 65
- 2025Retargeting Matters: General Motion Retargeting for Humanoid Motion Trackingcited by 60
- 2024Visual Whole-Body Control for Legged Loco-Manipulationcited by 58
- 2025AMO: Adaptive Motion Optimization for Hyper-Dexterous Humanoid Whole-Body Controlcited by 47
- 2024Mobile-TeleVision: Predictive Motion Priors for Humanoid Whole-Body Controlcited by 41
- 2025TWIST2: Scalable, Portable, and Holistic Humanoid Data Collection Systemcited by 39
- 2024FastUMI: A Scalable and Hardware-Independent Universal Manipulation Interface with Datasetcited by 36
- 2026Psi0: An Open Foundation Model Towards Universal Humanoid Loco-Manipulationcited by 25
- 2025MotionTrans: Human VR Data Enable Motion-Level Learning for Robotic Manipulation Policiescited by 22
- 2025HuB: Learning Extreme Humanoid Balancecited by 21
- 2026EgoHumanoid: Unlocking In-the-Wild Loco-Manipulation with Robot-Free Egocentric Demonstrationcited by 18
- 2025Behavior Foundation Model for Humanoid Robotscited by 15
- 2026Humanoid Manipulation Interface: Humanoid Whole-Body Manipulation from Robot-Free Demonstrationscited by 11
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026ω-0: A Latent Predictive World Action Model for Concurrent Humanoid Loco-Manipulationcited by 5
- 2026FWBC-VLA: Force-Aware Whole-Body Compensation for Contact-Rich Loco-Manipulationcited by 1
- 2026WholeBodyWAM: Generalizing Pre-trained World-Action Priors to Humanoid Loco-Manipulation via WBC-Grounded Coordinationcited by 0
- 2026Uni-VLaT: Whole-Body Tactile Adaptation of VLA Policies for Humanoid Loco-Manipulationcited by 0
- 2026Uni-VLaT: Whole-Body Tactile Adaptation of VLA Policies for Humanoid Loco-Manipulation28 Sep 2026
- 2026WholeBodyWAM: Generalizing Pre-trained World-Action Priors to Humanoid Loco-Manipulation via WBC-Grounded Coordination15 Sep 2026
- 2026FWBC-VLA: Force-Aware Whole-Body Compensation for Contact-Rich Loco-Manipulation3 Sep 2026
- 2026ω-0: A Latent Predictive World Action Model for Concurrent Humanoid Loco-Manipulation6 Aug 2026
Similar, not cited
Close in wording, no citation either way
- 2026Humanoid Loco-Manipulation With Discrete VLA Model
- 2025WholeBodyVLA: Towards Unified Latent VLA for Whole-Body Loco-Manipulation Control
- 2026WOLF-VLA: Whole-Body Humanoid Optimal Locomotion Framework for Vision-Language-Action Learning
- 2026Opt2VLA: Force-Aware Vision-Language-Action for Contact-Rich Humanoid Whole-Body Manipulation
- 2025TrajBooster: Boosting Humanoid Whole-Body Manipulation via Trajectory-Centric Learning
- 2026KINO: A Keyframe Interface for VLM Planning and Whole-Body Control in Humanoid Loco-Manipulation ↗
- 2025DemoHLM: From One Demonstration to Generalizable Humanoid Loco-Manipulation
- 2026LEGS: Fine-Tuning Teleop-Free VLAs for Humanoid Loco-manipulation in an Embodied Gaussian Splatting World