mlsanityData to 7 Oct 2026

Video Prediction Policy 2: Predict Better, Act Better

arXiv 2610.10270 ↗First posted 7 Oct 2026cs.CV cs.ROcited by 0

On the map: World models → Embodied and robot world models

also in 1 more
Abstract (arXiv)

World action models (WAMs) have emerged as an important class of generalist robot policies, aiming to transfer video prediction priors to action learning. However, we find that existing WAMs frequently produce incorrect motion predictions in open-ended environment, leading to erroneous actions. We attribute this limitation to two factors: (1) base video models are not optimized for manipulation, and (2) naively incorporating action components into video models can substantially degrade their generalization capabilities. We introduce Video Prediction Policy 2 (VPP2), a WAM that enables strong zero-shot generalization in both video prediction and action generation. First, we curate a large-scale, diverse dataset of manipulation videos to continue pretraining the base video foundation model. We annotate video clips with detailed captions and perform textitevent-level video pretraining to promote generalization across open-ended manipulation tasks. Second, we post-train and distill the video model into a single-step visual planner with fixed prediction horizon. Finally, we introduce action module via a mixture-of-transformers (MoT) architecture to learn implicit inverse dynamics model. Experiments demonstrate three key results: (1) VPP2-14B outperforms Cosmos3-64B by 11.0% points in video prediction instruction-following success rate on open-ended tasks; (2) VPP2 surpasses the strongest baseline by 18.5% points in success rate on real-world zero-shot ALOHA manipulation tasks; and (3) following benchmark-specific post-training, VPP2 achieves the highest success rates among evaluated methods on the challenging LIBERO-Pro, LIBERO-OOD, and RoboDojo benchmarks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 59 of its 77 references that are in the map. First posted on arXiv 7 Oct 2026.

Builds on

59 of 77 references.

Closest ancestors · 13

Same direction, newest first

  1. 2026OpenWAM: An Open, Modular Exploration Towards Systematic World-Action Model Pretraining7 Sep
  2. 2026GE-Act 2.0: Pretraining and Scaling a World-Action Model for Robotic Manipulation4 Sep
  3. 2026Robust-WAM: Bridging Generative Pretraining and Semantic Foresight in World-Action Models6 Aug
  4. 2026Understanding and Mitigating the Video-Action Generalization Gap via Temporal Ratio9 Jul
  5. 2026Qwen-RobotWorld Technical Report: Unifying Embodied World Modeling through Language-Conditioned Video Generation15 Jun
  6. 2026WALL-WM: Carving World Action Modeling at the Event Joints1 Jun
  7. 2026Motubrain: An Advanced World Action Model for Robot Control30 Apr
  8. 2026Fast-WAM: Do World Action Models Need Test-time Future Imagination?17 Mar
  9. 2026AtomVLA: Scalable Post-Training for Robotic Manipulation via Predictive Latent World Models9 Mar
  10. 2026World Action Models are Zero-shot Policies17 Feb
  11. 2026Causal World Modeling for Robot Control29 Jan
  12. 2025Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets3 Apr
1 more
  1. 2023Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Models16 Oct

Other references · 46

  1. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  2. 2025Qwen3-VL Technical Reportcited by 2,534
  3. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  4. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  5. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  6. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  7. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
  8. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  9. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
  10. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
  11. 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 670
  12. 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 390
34 more
  1. 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 332
  2. 2024Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representationscited by 279
  3. 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 274
  4. 2025Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Modelscited by 216
  5. 2025Unified Video Action Modelcited by 195
  6. 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 191
  7. 2025MolmoAct: Action Reasoning Models that can Reason in Spacecited by 189
  8. 2023RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shotcited by 173
  9. 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 166
  10. 2025LIBERO-PRO: Towards Robust and Fair Evaluation of Vision-Language-Action Models Beyond Memorizationcited by 166
  11. 2024Gen2Act: Human Video Generation in Novel Scenarios enables Generalizable Robot Manipulationcited by 142
  12. 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 137
  13. 2025Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulationcited by 129
  14. 2025Galaxea Open-World Dataset and G0 Dual-System VLA Modelcited by 90
  15. 2024Dreamitate: Real-World Visuomotor Policy Learning via Video Generationcited by 84
  16. 2025RoboCOIN: An Open-Sourced Bimanual Robotic Data Collection for Integrated Manipulationcited by 72
  17. 2026DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Controlcited by 71
  18. 2025Large Video Planner Enables Generalizable Robot Controlcited by 67
  19. 2025Scalable Vision-Language-Action Model Pretraining for Robotic Manipulation with Real-Life Human Activity Videoscited by 63
  20. 2025Vidar: Embodied Video Diffusion Model for Generalist Manipulationcited by 58
  21. 2026Seed2.0 Model Card: Towards Intelligence Frontier for Real-World Complexitycited by 48
  22. 2025InternData-A1: Pioneering High-Fidelity Synthetic Data for Pre-training Generalist Policycited by 46
  23. 2026Harness VLA: Steering Frozen VLAs into Reliable Manipulation Primitives via Memory-Guided Agentscited by 37
  24. 2018On the effectiveness of task granularity for transfer learningcited by 35
  25. 2026RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policiescited by 35
  26. 2026Xiaomi-Robotics-1: Scaling Vision-Language-Action Models with over 100K Hours of Real-World Trajectoriescited by 24
  27. 2026S-VAM: Shortcut Video-Action Model by Self-Distilling Geometric and Semantic Foresightcited by 16
  28. 2025Video2Act: A Dual-System Video Diffusion Policy with Robotic Spatio-Motional Modelingcited by 13
  29. 2025VLAs are Confined yet Capable of Generalizing to Novel Instructionscited by 8
  30. 2026An Unexpected Robot Policy: Early Evaluations of GPT-6 Astra on RoboDojo and Beyondcited by 8
  31. 2026Scalable Behavior Cloning with Open Data, Training, and Evaluationcited by 7
  32. 2026The Great March 100: 100 Detail-oriented Tasks for Evaluating Embodied AI Agentscited by 5
  33. 2026Veo-Act: Enhancing VLA Policies with Frontier Video Modelscited by 5
  34. 2026UAM: A Dual-Stream Perspective on Forgetting in VLA Trainingcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026τ0-WM: A Unified Video-Action World Model for Robotic Manipulation
  2. 2026InternW0-Δ: A World Action Model Bridging Predictive Dynamics and Actions with 20K+ Hours of Open Data
  3. 2026Say, Dream, and Act: Learning Video World Models for Instruction-Driven Robot Manipulation
  4. 2026Vid2WAM: Distilling Video Diffusion Priors into World Action Models
  5. 2026Zero-WAM: In-Context World-Action Modeling from Human Videos for Open-Ended Task Generalization
  6. 2025Scalable Policy Evaluation with Video World Models
  7. 2026GigaWorld-Policy: An Efficient Action-Centered World--Action Model
  8. 2026OpenWAM: An Open Framework for Composable World-Action Models