Learning Transferable World-Action Models from Task-Paired Human Videos
On the map: World models → Embodied and robot world models
Abstract (arXiv)
World-action models inherit broad visual and physical priors from video pre-training. However, standard within-video next-frame prediction allows them to rely on appearance shared between the context and future, rather than learning interactions that transfer across scenes and embodiments. We introduce WATCH, a cross-video pre-training that learns from task-paired human demonstrations collected across diverse environments. Conditioned on the video and actions of one demonstration, the model jointly predicts another demonstration's actions and future frames from its current observation. Predicting one demonstration from another encourages the model to learn task-relevant interactions that transfer across scenes and demonstrators. Our pre-training shows that these capabilities can be learned from readily scalable human-human pairs alone, without requiring matched human-robot demonstrations. Analyses before robot post-training show improved motion alignment across embodiments and better action and video prediction on unseen robots, with representations closer to those learned from robot pairs. After robot post-training, WATCH improves MimicDroid in-context success from 40.9% to 68.3% with Cosmos Policy and from 59.8% to 77.7% with Cosmos3, with the largest gains on the unseen levels. Gains persist under context-free post-training across robot embodiments. WATCH outperforms within-video pre-training and co-training baselines, supporting cross-video prediction as a scalable approach to learning transferable world-action representations.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 42 of its 46 references that are in the map. First posted on arXiv 9 Oct 2026.
Builds on
42 of 46 references.
Closest ancestors · 7
Same direction, newest first
- 2026ZimaBlue: Evolving Generalizable World Action Models through Scalable Video Pre-training31 Aug
- 2026Zero-WAM: In-Context World-Action Modeling from Human Videos for Open-Ended Task Generalization26 Aug
- 2026Fast-WAM: Do World Action Models Need Test-time Future Imagination?17 Mar
- 2026World Action Models are Zero-shot Policies17 Feb
- 2026Learning Latent Action World Models In The Wild8 Jan
- 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning11 Jun
- 2025Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets3 Apr
Other references · 35
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
- 2021Improving language models by retrieving from trillions of tokenscited by 1,293
- 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
- 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 407
- 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrationscited by 327
- 2024Latent Action Pretraining from Videoscited by 321
- 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 300
- 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 211
- 2025Unified Video Action Modelcited by 205
23 more
- 2024EgoMimic: Scaling Imitation Learning via Egocentric Videocited by 175
- 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 175
- 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 165
- 2025EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videoscited by 128
- 2023RoboVQA: Multimodal Long-Horizon Reasoning for Roboticscited by 120
- 2025Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videoscited by 102
- 2026EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Datacited by 94
- 2024In-Context Imitation Learning via Next-Token Predictioncited by 70
- 2026EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the Worldcited by 57
- 2024Vid2Robot: End-to-end Video-conditioned Policy Learning with Cross-Attention Transformerscited by 52
- 2025Emergence of Human to Robot Transfer in Vision-Language-Action Modelscited by 42
- 2024Retrieval-Augmented Diffusion Models for Time Series Forecastingcited by 39
- 2024R+X: Retrieval and Execution from Everyday Human Videoscited by 38
- 2022Semi-Parametric Neural Image Synthesiscited by 37
- 2025UniSkill: Imitating Human Videos via Cross-Embodiment Skill Representationscited by 35
- 2025EgoBridge: Domain Adaptation for Generalizable Imitation from Egocentric Human Datacited by 29
- 2025Human2Robot: Learning Robot Actions from Paired Human-Robot Videoscited by 25
- 2025Humanoid Everyday: A Comprehensive Robotic Dataset for Open-World Humanoid Manipulationcited by 25
- 2025MimicDroid: In-Context Learning for Humanoid Robot Manipulation from Human Play Videoscited by 20
- 2024ScrewMimic: Bimanual Imitation from Human Videos with Screw Space Projectioncited by 19
- 2025TwinVLA: Data-Efficient Bimanual Manipulation with Twin Single-Arm Vision-Language-Action Modelscited by 13
- 2025HAND Me the Data: Fast Robot Adaptation via Hand Path Retrievalcited by 10
- 2026Retrieve, Don't Retrain: Extending Vision Language Action Models to New Tasks at Test Timecited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Learning Transferable Dynamics Priors from Action to World Modeling
- 2026Native Action-Prior Learning from Videos for World Action Models
- 2025TraceGen: World Modeling in 3D Trace Space Enables Learning from Cross-Embodiment Videos
- 2026CLAP: Cross-Embodiment Video World Models are Zero-Shot Physical Simulators
- 2026From Human Videos to Robot Manipulation: A Survey on Scalable Vision-Language-Action Learning with Human-Centric Data
- 2026LD4WAM: Learning Latent Dynamics from Human Videos for World Action Models
- 2026InternW0-Δ: A World Action Model Bridging Predictive Dynamics and Actions with 20K+ Hours of Open Data
- 2026Being-M0.7: A Latent World-Action Model for Humanoid Robots