# World2Motion: Turning Video World Models into 3D Human Motion Generators

arXiv: https://arxiv.org/abs/2609.37004 · HTML: https://mlsanity.com/paper/2609.37004/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Video-generation world models and simulators, one of 11 directions in World models.
- Builds on 11 of its 53 references that are in the map.
- First posted on arXiv 29 Sep 2026.

## Placed in

- World models / [Video-generation world models and simulators](https://mlsanity.com/wm/video-generation-world-models-and-simulators/)

## Builds on (most cited first, 11 of 11)

- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [Cosmos 3: Omnimodal World Models for Physical AI](https://mlsanity.com/paper/2606.02800/) (2026)
- [Kimodo: Scaling Controllable Human Motion Generation](https://mlsanity.com/paper/2603.15546/) (2026)
- [HY-Motion 1.0: Scaling Flow Matching Models for Text-To-Motion Generation](https://mlsanity.com/paper/2512.23464/) (2025)
- [CoMoVi: Co-Generation of 3D Human Motions and Realistic Videos](https://mlsanity.com/paper/2601.10632/) (2026)
- [OnlineHMR: Video-based Online World-Grounded Human Mesh Recovery](https://mlsanity.com/paper/2603.17355/) (2026)
- [InfBaGel: Human-Object-Scene Interaction Generation with Dynamic Perception and Iterative Refinement](https://mlsanity.com/paper/2604.04843/) (2026)
- [GIRAF: Towards Generalizable Human Interactions with Articulated Objects](https://mlsanity.com/paper/2607.07880/) (2026)

## Abstract (arXiv)

We present World2Motion, a framework that generates scene-aware 3D human motion and corresponding video from a single image and a text prompt. While existing 3D motion generators learn from motion datasets, their generalization is constrained by limited coverage of environments. In contrast, video world models such as Cosmos 3 offer broader environmental priors but are not designed for full-body motion generation; recovering motion from their generated videos requires costly two-stage inference. To address these, we turn Cosmos 3 into a single-stage 3D motion generator. This adaptation has two challenges: the scarcity of paired video--motion data and temporal instability in the generated motion. First, we construct a training dataset combining synthetic video--motion pairs with real videos paired with estimated 3D motion. Second, we propose a shift-decoupled noise schedule that assigns different noise levels to video and motion through shared denoising progress. This design accommodates the different denoising requirements of the two modalities, reducing motion jitter. Experiments on a multi-source interaction benchmark show that World2Motion has better motion--text alignment and scene interaction compared with the evaluated 3D motion generators. It also matches the interaction success rate of the two-stage baseline while achieving approximately 3.3× faster inference. Our project page is available at https://fyantu.github.io/World2Motion/.