VIDEAS: Distilling Explicit Action Semantics from Demonstration Videos for World Models via Prior-Guided Simulation
On the map: World models → Embodied and robot world models also fits: Latent predictive world models
Abstract (arXiv)
World models learn internal representations of environment dynamics to predict future states, enabling agents to optimize action plans without physical interactions. However, developing world models that genuinely internalize underlying causal physical laws to explicitly reason about action preconditions and subsequent state transitions remains an open challenge. In this paper, we propose VIDEAS, a data distillation framework that transforms continuous physical dynamics from operational videos into explicit action semantics for foundation models. Specifically, it deconstructs visual demonstrations into discrete action trajectories and utilizes advanced vision-language models (VLMs) to extract structured knowledge encapsulating action preconditions and effects. To ensure physical consistency, we introduce a prior-guided trajectory simulation mechanism grounded within a text-based environment to rigorously validate the extracted knowledge. Notably, we incorporate negative trajectories to enrich knowledge completeness and enhance data diversity to mitigate cognitive bias. Furthermore, we present VIDEAS-WM, an 8B/9B-parameter suite of language-based world models trained on 34K high-quality samples derived from AgiBot-World dataset. Extensive experiments demonstrate that VIDEAS-WM establishes state-of-the-art performance in high-level embodied action semantic reasoning, exhibiting profound physical understanding and robust generalization across unseen scenarios.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 17 of its 46 references that are in the map. First posted on arXiv 27 Sep 2026.
Builds on
17 of 46 references.
Closest ancestors · 3
Same direction, newest first
- 2025Planning with Reasoning using Vision Language World Model2 Sep
- 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning11 Jun
- 2025WorldPrediction: A Benchmark for High-level World Modeling and Long-horizon Procedural Planning4 Jun
Other references · 14
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2025Qwen3 Technical Reportcited by 8,195
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2018World Modelscited by 1,444
- 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 390
- 2024Gen2Act: Human Video Generation in Novel Scenarios enables Generalizable Robot Manipulationcited by 142
- 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 137
- 2024Learning and Leveraging World Models in Visual Representation Learningcited by 71
- 2025From Word to World: Can Large Language Models be Implicit Text-based World Models?cited by 40
- 2025Dyn-O: Building Structured World Models with Object-Centric Representationscited by 12
- 2025Learning Interactive World Model for Object-Centric Reinforcement Learningcited by 11
- 2025One Demo Is All It Takes: Planning Domain Derivation with LLMs from A Single Demonstrationcited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026X-Foresight: A Joint Vision-Action Causal Forecasting Network via Predictive World Modeling
- 2026InternW0-Δ: A World Action Model Bridging Predictive Dynamics and Actions with 20K+ Hours of Open Data
- 2026World Action Models: The Next Frontier in Embodied AI
- 2026OpenWAM: An Open, Modular Exploration Towards Systematic World-Action Model Pretraining
- 2026World Tokens: Enhancing Embodied Policies with Training-Time World Modeling
- 2026Vid2WAM: Distilling Video Diffusion Priors into World Action Models
- 2026Causal World Modeling for Robot Control
- 2026Decoupling Intention from Trajectory: A Representational Deduction Framework for World Action Models