# RynnWorld-4D: 4D Embodied World Models for Robotic Manipulation arXiv: https://arxiv.org/abs/2607.06559 · HTML: https://mlsanity.com/paper/2607.06559/ ## Key facts - Cited by 7 papers in the map (data through 14 Oct 2026). - Placed in Embodied and robot world models, one of 11 directions in World models. - Builds on 28 of its 84 references that are in the map. - First posted on arXiv 7 Jul 2026. ## Placed in - World models / [Embodied and robot world models](https://mlsanity.com/wm/embodied-and-robot-world-models/) ## Builds on (most cited first, 28 of 28) - [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022) - [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023) - [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025) - [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024) - [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024) - [π0.5: a Vision-Language-Action Model with Open-World Generalization](https://mlsanity.com/paper/2504.16054/) (2025) - [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019) - [World Models](https://mlsanity.com/paper/1803.10122/) (2018) - [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018) - [Cosmos World Foundation Model Platform for Physical AI](https://mlsanity.com/paper/2501.03575/) (2025) - [Recurrent World Models Facilitate Policy Evolution](https://mlsanity.com/paper/1809.01999/) (2018) - [Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models](https://mlsanity.com/paper/1805.12114/) (2018) - [RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation](https://mlsanity.com/paper/2410.07864/) (2024) - [Open-Sora: Democratizing Efficient Video Production for All](https://mlsanity.com/paper/2412.20404/) (2024) - [TD-MPC2: Scalable, Robust World Models for Continuous Control](https://mlsanity.com/paper/2310.16828/) (2023) - [World Action Models are Zero-shot Policies](https://mlsanity.com/paper/2602.15922/) (2026) - [Latent Action Pretraining from Videos](https://mlsanity.com/paper/2410.11758/) (2024) - [Causal World Modeling for Robot Control](https://mlsanity.com/paper/2601.21998/) (2026) - [Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Models](https://mlsanity.com/paper/2310.10639/) (2023) - [Motus: A Unified Latent Action World Model](https://mlsanity.com/paper/2512.13030/) (2025) - [RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulation](https://mlsanity.com/paper/2412.13877/) (2024) - [CAT4D: Create Anything in 4D with Multi-View Video Diffusion Models](https://mlsanity.com/paper/2411.18613/) (2024) - [mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAs](https://mlsanity.com/paper/2512.15692/) (2025) - [Training Agents Inside of Scalable World Models](https://mlsanity.com/paper/2509.24527/) (2025) - [Galaxea Open-World Dataset and G0 Dual-System VLA Model](https://mlsanity.com/paper/2509.00576/) (2025) - [Pandora: Towards General World Model with Natural Language Actions and Video States](https://mlsanity.com/paper/2406.09455/) (2024) - [4Real: Towards Photorealistic 4D Scene Generation via Video Diffusion Models](https://mlsanity.com/paper/2406.07472/) (2024) - [LARY: A Latent Action Representation Yielding Benchmark for Generalizable Vision-to-Action Alignment](https://mlsanity.com/paper/2604.11689/) (2026) ## Built on by (most cited first, 7 of 7) - [Xiaomi-Robotics-1: Scaling Vision-Language-Action Models with over 100K Hours of Real-World Trajectories](https://mlsanity.com/paper/2607.15330/) (2026) - [4D-WAM: Infusing Spatiotemporal Awareness into World Action Models through Trajectory Fields](https://mlsanity.com/paper/2608.08023/) (2026) - [Copper-Policy: Focus on the Representation for Robust Robot Manipulation](https://mlsanity.com/paper/2609.32779/) (2026) - [Magic-W0: A Structured World-Action Foundation Model for Physical Intelligence](https://mlsanity.com/paper/2609.39870/) (2026) - [XGenAct: Geometry-Enhanced World Action Models through Cross-Task Generation](https://mlsanity.com/paper/2610.03516/) (2026) - [SimForcing: Distilling Simulation Motion Priors into Real-Domain Robot World Models](https://mlsanity.com/paper/2610.06598/) (2026) - [Thinker-WM: A Unified World Model with Shared Multimodal Representations for Embodied Intelligence](https://mlsanity.com/paper/2610.18447/) (2026) ## Abstract (arXiv) Robotic manipulation in the open world requires not only recognizing what a scene looks like, but also anticipating how its 3D structure moves under interaction. We argue that synchronized RGB, depth, and optical flow (RGB-DF) provide a physically grounded representation that captures the underlying 4D dynamics of a scene. Compared to 2D pixel videos, this multi-modal synergy aligns visual appearance with geometric structure and temporal motion, creating a representation space significantly closer to low-level end-effector actions demanded by robotic systems, narrowing the gap between world prediction and policy learning. Building on this insight, we introduce RynnWorld-4D, a generative model that co-produces future RGB frames, depth maps, and optical flow from a single RGB-D image and a language instruction within one unified diffusion process. This 4D world model features a tri-branch architecture that integrates cross-modal attention with frame-wise 3D RoPE, ensuring that appearance, geometry, and motion evolve consistently. To supply training data at scale, we curate Rynn4DDataset 1.0, a massive dataset of over 254.4 million frames across egocentric human and robotic manipulation videos with high-quality pseudo-labels for depth and optical flow. We further propose RynnWorld-4D-Policy, an inverse dynamics head that consumes the internal 4D representations of RynnWorld-4D in a single forward pass, bypassing expensive multi-step denoising, to output robot actions in a closed-loop manner. Experiments show that RynnWorld-4D produces temporally and spatially coherent 4D predictions, and that RynnWorld-4D-Policy achieves state-of-the-art performance on real-world dexterous bimanual manipulation tasks, particularly excelling in tasks demanding spatial precision and temporal coordination.