# From Language to Locomotion: Retargeting-free Humanoid Control via Motion Latent Guidance

arXiv: https://arxiv.org/abs/2510.14952 · HTML: https://mlsanity.com/paper/2510.14952/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 52 references that are in the map.
- First posted on arXiv 16 Oct 2025.

## Builds on (most cited first, 30 of 44)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [MaskGIT: Masked Generative Image Transformer](https://mlsanity.com/paper/2202.04200/) (2022)
- [Human Motion Diffusion Model](https://mlsanity.com/paper/2209.14916/) (2022)
- [SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers](https://mlsanity.com/paper/2401.08740/) (2024)
- [Autoregressive Image Generation without Vector Quantization](https://mlsanity.com/paper/2406.11838/) (2024)
- [T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representations](https://mlsanity.com/paper/2301.06052/) (2023)
- [Executing your Commands via Motion Diffusion in Latent Space](https://mlsanity.com/paper/2212.04048/) (2022)
- [MotionGPT: Human Motion as a Foreign Language](https://mlsanity.com/paper/2306.14795/) (2023)
- [MoMask: Generative Masked Modeling of 3D Human Motions](https://mlsanity.com/paper/2312.00063/) (2023)
- [TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Texts](https://mlsanity.com/paper/2207.01696/) (2022)
- [ReMoDiffuse: Retrieval-Augmented Motion Diffusion Model](https://mlsanity.com/paper/2304.01116/) (2023)
- [Perpetual Humanoid Control for Real-time Simulated Avatars](https://mlsanity.com/paper/2305.06456/) (2023)
- [FLAME: Free-form Language-based Motion Synthesis & Editing](https://mlsanity.com/paper/2209.00349/) (2022)
- [OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning](https://mlsanity.com/paper/2406.08858/) (2024)
- [HumanPlus: Humanoid Shadowing and Imitation from Humans](https://mlsanity.com/paper/2406.10454/) (2024)
- [MotionGPT: Finetuned LLMs Are General-Purpose Motion Generators](https://mlsanity.com/paper/2306.10900/) (2023)
- [RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concrete](https://mlsanity.com/paper/2502.21257/) (2025)
- [ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills](https://mlsanity.com/paper/2502.01143/) (2025)
- [RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Robotics](https://mlsanity.com/paper/2506.04308/) (2025)
- [AttT2M: Text-Driven Human Motion Generation with Multi-Perspective Attention Mechanism](https://mlsanity.com/paper/2309.00796/) (2023)
- [3D Human Pose Estimation via Intuitive Physics](https://mlsanity.com/paper/2303.18246/) (2023)
- [HOVER: Versatile Neural Whole-Body Controller for Humanoid Robots](https://mlsanity.com/paper/2410.21229/) (2024)
- [RoboBrain 2.0 Technical Report](https://mlsanity.com/paper/2507.02029/) (2025)
- [ExBody2: Advanced Expressive Humanoid Whole-Body Control](https://mlsanity.com/paper/2412.13196/) (2024)
- [MotionStreamer: Streaming Motion Generation via Diffusion-based Autoregressive Model in Causal Latent Space](https://mlsanity.com/paper/2503.15451/) (2025)
- [Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregression](https://mlsanity.com/paper/2411.16575/) (2024)

## Built on by (most cited first, 18 of 18)

- [TextOp: Real-time Interactive Text-Driven Humanoid Robot Motion Generation and Control](https://mlsanity.com/paper/2602.07439/) (2026)
- [FRoM-W1: Towards General Humanoid Whole-Body Control with Language Instructions](https://mlsanity.com/paper/2601.12799/) (2026)
- [HEX: Humanoid-Aligned Experts for Cross-Embodiment Whole-Body Manipulation](https://mlsanity.com/paper/2604.07993/) (2026)
- [ZeroWBC: Learning Natural Whole-Body Humanoid Interaction from Human Egocentric Data](https://mlsanity.com/paper/2603.09170/) (2026)
- [RoboMirror: Understand Before You Imitate for Video to Humanoid Locomotion](https://mlsanity.com/paper/2512.23649/) (2025)
- [ω-0: A Latent Predictive World Action Model for Concurrent Humanoid Loco-Manipulation](https://mlsanity.com/paper/2608.06375/) (2026)
- [UniAct: Unified Motion Generation and Action Streaming for Humanoid Robots](https://mlsanity.com/paper/2512.24321/) (2025)
- [Heracles: Bridging Precise Tracking and Generative Synthesis for General Humanoid Control](https://mlsanity.com/paper/2603.27756/) (2026)
- [Action-Sketcher: From Reasoning to Action via Visual Sketches for Long-Horizon Robotic Manipulation](https://mlsanity.com/paper/2601.01618/) (2026)
- [ECHO: Edge-Cloud Humanoid Orchestration for Language-to-Motion Control](https://mlsanity.com/paper/2603.16188/) (2026)
- [PhysMoDPO: Physically-Plausible Humanoid Motion with Preference Optimization](https://mlsanity.com/paper/2603.13228/) (2026)
- [PhyGile: Physics-Prefix Guided Motion Generation for Agile General Humanoid Motion Tracking](https://mlsanity.com/paper/2603.19305/) (2026)
- [GenTrack: Physical Alignment for Robot-Native Motion Generation and Zero-Shot Humanoid Tracking](https://mlsanity.com/paper/2608.01410/) (2026)
- [MIND: Multi-Scale Intent Diffusion for Text-Driven Physics-Based Humanoid Control](https://mlsanity.com/paper/2605.26006/) (2026)
- [Before the Body Moves: Learning Anticipatory Joint Intent for Language-Conditioned Humanoid Control](https://mlsanity.com/paper/2605.14417/) (2026)
- [Human-Centric Intelligence in the Era of Foundation Models: A Survey](https://mlsanity.com/paper/2608.18184/) (2026)
- [PredActor: Predictive Action Diffusion for Steerable Onboard Humanoid Control](https://mlsanity.com/paper/2609.24840/) (2026)
- [Sample, Simulate, Select: Physics-in-the-Loop Text-to-Motion for Humanoids Without Training](https://mlsanity.com/paper/2609.26420/) (2026)

## Abstract (arXiv)

Natural language offers a natural interface for humanoid robots, but existing language-guided humanoid locomotion pipelines remain cumbersome and untrustworthy. They typically decode human motion, retarget it to robot morphology, and then track it with a physics-based controller. However, this multi-stage process is prone to cumulative errors, introduces high latency, and yields weak coupling between semantics and control. These limitations call for a more direct pathway from language to action, one that eliminates fragile intermediate stages. Therefore, we present RoboGhost, a retargeting-free framework that directly conditions humanoid policies on language-grounded motion latents. By bypassing explicit motion decoding and retargeting, RoboGhost enables a diffusion-based policy to denoise executable actions directly from noise, preserving semantic intent and supporting fast, reactive control. A hybrid causal transformer-diffusion motion generator further ensures long-horizon consistency while maintaining stability and diversity, yielding rich latent representations for precise humanoid behavior. Extensive experiments demonstrate that RoboGhost substantially reduces deployment latency, improves success rates and tracking precision, and produces smooth, semantically aligned locomotion on real humanoids. Beyond text, the framework naturally extends to other modalities such as images, audio, and music, providing a universal foundation for vision-language-action humanoid systems.