mlsanityData to 7 Oct 2026

VIDEAS: Distilling Explicit Action Semantics from Demonstration Videos for World Models via Prior-Guided Simulation

arXiv 2609.33464 ↗First posted 27 Sep 2026cs.ROcited by 0

On the map: World models → Embodied and robot world models also fits: Latent predictive world models

Abstract (arXiv)

World models learn internal representations of environment dynamics to predict future states, enabling agents to optimize action plans without physical interactions. However, developing world models that genuinely internalize underlying causal physical laws to explicitly reason about action preconditions and subsequent state transitions remains an open challenge. In this paper, we propose VIDEAS, a data distillation framework that transforms continuous physical dynamics from operational videos into explicit action semantics for foundation models. Specifically, it deconstructs visual demonstrations into discrete action trajectories and utilizes advanced vision-language models (VLMs) to extract structured knowledge encapsulating action preconditions and effects. To ensure physical consistency, we introduce a prior-guided trajectory simulation mechanism grounded within a text-based environment to rigorously validate the extracted knowledge. Notably, we incorporate negative trajectories to enrich knowledge completeness and enhance data diversity to mitigate cognitive bias. Furthermore, we present VIDEAS-WM, an 8B/9B-parameter suite of language-based world models trained on 34K high-quality samples derived from AgiBot-World dataset. Extensive experiments demonstrate that VIDEAS-WM establishes state-of-the-art performance in high-level embodied action semantic reasoning, exhibiting profound physical understanding and robust generalization across unseen scenarios.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 17 of its 46 references that are in the map. First posted on arXiv 27 Sep 2026.

Builds on

17 of 46 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Planning with Reasoning using Vision Language World Model2 Sep
  2. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning11 Jun
  3. 2025WorldPrediction: A Benchmark for High-level World Modeling and Long-horizon Procedural Planning4 Jun

Other references · 14

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2025Qwen2.5-VL Technical Reportcited by 5,418
  4. 2018World Modelscited by 1,444
  5. 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 390
  6. 2024Gen2Act: Human Video Generation in Novel Scenarios enables Generalizable Robot Manipulationcited by 142
  7. 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 137
  8. 2024Learning and Leveraging World Models in Visual Representation Learningcited by 71
  9. 2025From Word to World: Can Large Language Models be Implicit Text-based World Models?cited by 40
  10. 2025Dyn-O: Building Structured World Models with Object-Centric Representationscited by 12
  11. 2025Learning Interactive World Model for Object-Centric Reinforcement Learningcited by 11
  12. 2025One Demo Is All It Takes: Planning Domain Derivation with LLMs from A Single Demonstrationcited by 5
2 more
  1. 2025UniDomain: Pretraining a Unified PDDL Domain from Real-World Demonstrations for Generalizable Robot Task Planningcited by 5
  2. 2026LogicEnvGen: Task-Logic Driven Generation of Diverse Simulated Environments for Embodied AIcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026X-Foresight: A Joint Vision-Action Causal Forecasting Network via Predictive World Modeling
  2. 2026InternW0-Δ: A World Action Model Bridging Predictive Dynamics and Actions with 20K+ Hours of Open Data
  3. 2026World Action Models: The Next Frontier in Embodied AI
  4. 2026OpenWAM: An Open, Modular Exploration Towards Systematic World-Action Model Pretraining
  5. 2026World Tokens: Enhancing Embodied Policies with Training-Time World Modeling
  6. 2026Vid2WAM: Distilling Video Diffusion Priors into World Action Models
  7. 2026Causal World Modeling for Robot Control
  8. 2026Decoupling Intention from Trajectory: A Representational Deduction Framework for World Action Models