# τ0-WM: A Unified Video-Action World Model for Robotic Manipulation

arXiv: https://arxiv.org/abs/2606.01027 · HTML: https://mlsanity.com/paper/2606.01027/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and robot world models, one of 11 directions in World models.
- Builds on 23 of its 41 references that are in the map.
- First posted on arXiv 31 May 2026.

## Placed in

- World models / [Embodied and robot world models](https://mlsanity.com/wm/embodied-and-robot-world-models/)

## Builds on (most cited first, 23 of 23)

- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [π0.5: a Vision-Language-Action Model with Open-World Generalization](https://mlsanity.com/paper/2504.16054/) (2025)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [WorldSimBench: Towards Video Generation Models as World Simulators](https://mlsanity.com/paper/2410.18072/) (2024)
- [Cosmos World Foundation Model Platform for Physical AI](https://mlsanity.com/paper/2501.03575/) (2025)
- [LTX-Video: Realtime Video Latent Diffusion](https://mlsanity.com/paper/2501.00103/) (2024)
- [World Action Models are Zero-shot Policies](https://mlsanity.com/paper/2602.15922/) (2026)
- [Learning Interactive Real-World Simulators](https://mlsanity.com/paper/2310.06114/) (2023)
- [DayDreamer: World Models for Physical Robot Learning](https://mlsanity.com/paper/2206.14176/) (2022)
- [Fast-WAM: Do World Action Models Need Test-time Future Imagination?](https://mlsanity.com/paper/2603.16666/) (2026)
- [Causal World Modeling for Robot Control](https://mlsanity.com/paper/2601.21998/) (2026)
- [Diffusion for World Modeling: Visual Details Matter in Atari](https://mlsanity.com/paper/2405.12399/) (2024)
- [Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planning](https://mlsanity.com/paper/2601.16163/) (2026)
- [Motus: A Unified Latent Action World Model](https://mlsanity.com/paper/2512.13030/) (2025)
- [Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets](https://mlsanity.com/paper/2504.02792/) (2025)
- [Ctrl-World: A Controllable Generative World Model for Robot Manipulation](https://mlsanity.com/paper/2510.10125/) (2025)
- [TransDreamer: Reinforcement Learning with Transformer World Models](https://mlsanity.com/paper/2202.09481/) (2022)
- [DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos](https://mlsanity.com/paper/2602.06949/) (2026)
- [EnerVerse: Envisioning Embodied Future Space for Robotics Manipulation](https://mlsanity.com/paper/2501.01895/) (2025)

## Built on by (most cited first, 27 of 27)

- [Zero-WAM: In-Context World-Action Modeling from Human Videos for Open-Ended Task Generalization](https://mlsanity.com/paper/2608.26103/) (2026)
- [World Action Models: A Survey](https://mlsanity.com/paper/2606.20781/) (2026)
- [GigaWorld-1: A Roadmap to Build World Models for Robot Policy Evaluation](https://mlsanity.com/paper/2607.02642/) (2026)
- [SelfWAM: A Self-Grounded Unified World Action Model for Fast Robot Control](https://mlsanity.com/paper/2608.00725/) (2026)
- [Test-Time Scaling for World Action Models via Zero-Shot Geometric Evaluation](https://mlsanity.com/paper/2607.17454/) (2026)
- [FACT: Failure-Aware Causal Training for World-Action Models](https://mlsanity.com/paper/2608.10232/) (2026)
- [GE-Act 2.0: Pretraining and Scaling a World-Action Model for Robotic Manipulation](https://mlsanity.com/paper/2609.05588/) (2026)
- [Riemann-1.0: An Embodied World Action Model for Physical AI](https://mlsanity.com/paper/2608.27033/) (2026)
- [The Geometry of Flow-Matching Uncertainty: A Cost-free Uncertainty Proxy and Its Application in Flow-based VLA Failure Detection](https://mlsanity.com/paper/2607.27933/) (2026)
- [Vid2WAM: Distilling Video Diffusion Priors into World Action Models](https://mlsanity.com/paper/2608.08558/) (2026)
- [DELE-w0.5: Inferring Action from Future Latent State for Robotic Manipulation](https://mlsanity.com/paper/2608.22067/) (2026)
- [World Models for Embodied Intelligence: From Plausible to Controllable to Actionable](https://mlsanity.com/paper/2609.16697/) (2026)
- [XPACE: Joint World and Action Modeling from Heterogeneous Experience](https://mlsanity.com/paper/2609.17372/) (2026)
- [DualManip: Agentic Dynamic Manipulation via Dual-Path Semantic Reasoning and Geometric Adaptation](https://mlsanity.com/paper/2609.31112/) (2026)
- [From Generation to Simulation: How Far Are World Models from Being True Simulators?](https://mlsanity.com/paper/2608.23070/) (2026)
- [WholeBodyWAM: Learning Whole-Body World Action Models with Scalable Motion Priors](https://mlsanity.com/paper/2609.18197/) (2026)
- [ME-Dex 1.0: Bringing Heterogeneous Tactile Sensing into World Action Modeling](https://mlsanity.com/paper/2609.21449/) (2026)
- [Beyond Visual Quality: A Study of Test-Time Planning with World Action Models](https://mlsanity.com/paper/2609.24745/) (2026)
- [Efficient World Action Model Inference with Adaptive Intermediate States](https://mlsanity.com/paper/2609.34608/) (2026)
- [Spotter: Let the Embodied Model Lead, and the VLM Reflect for It](https://mlsanity.com/paper/2609.36808/) (2026)
- [SplineWAM: Adaptive Action Horizons for World Action Models via B-Spline Representations](https://mlsanity.com/paper/2609.39873/) (2026)
- [XGenAct: Geometry-Enhanced World Action Models through Cross-Task Generation](https://mlsanity.com/paper/2610.03516/) (2026)
- [Vela: Scaling Vision-Language-Action Models with Adaptive Action Curve Parametrization](https://mlsanity.com/paper/2610.05230/) (2026)
- [Future Anchored Verification and Online Recovery for World Action Models](https://mlsanity.com/paper/2610.06280/) (2026)
- [KineWorld: Action-Induced Transport Fields for Embodied World Modeling](https://mlsanity.com/paper/2610.06349/) (2026)
- [OpenWAM: An Open Framework for Composable World-Action Models](https://mlsanity.com/paper/2610.07922/) (2026)
- [LoopWAM: Closed-Loop Generalist Policy Improvement from Deployment Rollouts](https://mlsanity.com/paper/2610.18873/) (2026)

## Abstract (arXiv)

Robotic manipulation requires models that generate executable actions while anticipating and evaluating their future consequences before physical execution. We present τ0-World Model (τ0-WM), a unified video-action world model that integrates policy learning, video prediction, and action evaluation within a single future-predictive framework. Built on a shared video diffusion backbone, τ0-WM provides two complementary interfaces. First, a video action model jointly predicts future visual latents and continuous action chunks from multi-view observations, language instructions, and robot state. Second, an action-conditioned video simulator rolls out candidate action chunks into multi-view futures and predicts dense task-progress scores. The model is trained on approximately 27,300 hours of real-robot teleoperation, UMI-style interaction, egocentric human videos, and rollout or failure trajectories using modality-specific supervision masks. At inference time, τ0-WM uses test-time computation to sample action candidates, rank them with re-denoising consistency, and invoke simulator-based rectification for low-quality candidates. On challenging long-horizon and fine-grained robotic manipulation tasks, τ0-WM shows superior performance over other relevant baselines.