mlsanityData to 7 Oct 2026

Programmable World Model

arXiv 2609.10540 ↗First posted 9 Sep 2026cs.CVcited by 7

On the map: World models → Video-generation world models and simulators

Abstract (arXiv)

Recent video world models generate increasingly realistic and interactive visual experiences, yet lack reliable mechanisms for maintaining persistent world state and enforcing programmable rules over extended interactions. We introduce Programmable World Model, a framework that decouples world-state evolution from visual observation generation. An agent translates natural-language instructions into executable programs that specify entity states and state-transition rules, enabling direct control over individual entities and their interactions. A lightweight engine executes these programs to update and maintain an explicit, persistent global world state, including off-screen entities and non-visual attributes. To connect world state with visual generation, we introduce state-augmented 3D oriented bounding boxes (OBBs) as an intermediate representation. This representation, together with the target camera trajectory, is deterministically compiled into pixel-aligned spatiotemporal conditioning signals for a pretrained video model serving as the generative renderer. This design allows users to create playable games with predefined mechanics, direct control over individual entities, and persistent world state throughout gameplay. We further introduce CombatStateBench, a benchmark for evaluating programmable world models. On CombatStateBench, our method achieves 94% Count Accuracy and 98% State Accuracy, substantially outperforming existing interactive video world models while supporting coherent long-horizon generation. These results demonstrate the effectiveness of separating explicit state evolution from generative rendering for building persistent, programmable worlds.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 16 of its 17 references that are in the map. First posted on arXiv 9 Sep 2026.

Builds on

16 of 17 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026AlayaWorld: Interactive Long-Horizon World Modeling -- Full Technical Report20 Jul
  2. 2026Advancing Open-source World Models28 Jan
  3. 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling16 Dec
  4. 2025Yume: An Interactive World Generation Model23 Jul
  5. 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness27 Mar
  6. 2025Wan: Open and Advanced Large-Scale Video Generative Models26 Mar

Other references · 10

  1. 2025Qwen3-VL Technical Reportcited by 2,534
  2. 2024LTX-Video: Realtime Video Latent Diffusioncited by 549
  3. 2025ViPE: Video Pose Engine for 3D Geometric Perceptioncited by 162
  4. 2025DiffusionRenderer: Neural Inverse and Forward Rendering with Video Diffusion Modelscited by 55
  5. 2025VLIPP: Towards Physically Plausible Video Generation with Vision and Language Informed Physical Priorcited by 49
  6. 2026Infinite Worlds with Versatile Interactionscited by 37
  7. 2025RenderFormer: Transformer-based Neural Rendering of Triangle Meshes with Global Illuminationcited by 23
  8. 2026WildDet3D: Scaling Promptable 3D Detection in the Wildcited by 12
  9. 2026Coarse-to-Real: Generative Rendering for Populated Dynamic Scenescited by 8
  10. 2026Generative World Renderercited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 7.

7 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends
  2. 2026StatePlay: State-Aware Game World Models for Mechanics-Consistent Generation
  3. 2026From Pixels to States: Rethinking Interactive World Models as Game Engines
  4. 2026Code World Model: Coding Agent as World Brain
  5. 2026AlayaWorld: Long-Horizon and Playable Video World Generation
  6. 2026GameWAM: A World Action Model for Video Games
  7. 2026DreamX-World 1.0: A General-Purpose Interactive World Model
  8. 2026WildWorld: A Large-Scale Dataset for Dynamic World Modeling with Actions and Explicit State toward Generative ARPG