mlsanityData to 14 Oct 2026

Code Plans, Diffusion Renders: Open-Ended Generative World Modeling

arXiv 2609.26458 ↗First posted 22 Sep 2026cs.CVcited by 0

On the map: World models → Video-generation world models and simulators

Abstract (arXiv)

We introduce CoDeR, a new paradigm for world modeling. Unlike existing video world models that implicitly represent world dynamics through visual observations, our system explicitly constructs an executable world with code and employs video generation models for visual realization. Specifically, we coordinate five complementary roles to translate high-level concepts into structured world rules, executable dynamics, and perceptual observations. This design enables textitlong-term memory, textitopen-ended interactions, textitautonomous world evolution, and textitmulti-agent scenarios, where multiple entities can act, interact, and evolve persistently beyond the current observation. Extensive experiments demonstrate that our framework substantially extends the capabilities of existing world models, enabling long-term memory, open-ended interactions, autonomous evolution, and persistent multi-agent dynamics, while achieving state-of-the-art performance across multiple evaluation settings. Code and model weights will be made publicly available. Project Page: hrefhttps://becauseimbatman0.github.io/CoDeRCoDeR.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 98 of its 108 references that are in the map. First posted on arXiv 22 Sep 2026.

Builds on

98 of 108 references.

Closest ancestors · 53

Same direction, newest first

  1. 2026Matrix-Game 3.5: Enhancing Real-Time Streaming Interactive World Models with Patch Memory30 Aug
  2. 2026ReWorld: An Interactive World Model with Long-Horizon Memory24 Aug
  3. 2026Marionette: Predicting World States, Rendering Geometry, Painting Appearance14 Aug
  4. 2026AlayaWorld: Interactive Long-Horizon World Modeling - Full Technical Report (v1.1)13 Aug
  5. 2026Population-Scalable Multi-Agent World Modeling9 Aug
  6. 2026MASS: Multiplayer World Models with Authoritative Shared State6 Aug
  7. 2026Wonder: Video World Model Done Better28 Jul
  8. 2026Streaming Multi-Agent Autoregressive Diffusion Model with World State Registers23 Jul
  9. 2026ABot-World-0: Infinite Interactive World Rollout on a Single Desktop GPU21 Jul
  10. 2026Multiplayer Interactive World Models with Representation Autoencoders6 Jul
  11. 2026MemLearner: Learning to Query Context memory for Video World Models30 Jun
  12. 2026ActWorld: From Explorable to Interactive World Model via Action-Aware Memory16 Jun
38 more
  1. 2026DreamX-World 1.0: A General-Purpose Interactive World Model15 Jun
  2. 2026MoVerse: Real-Time Video World Modeling with Panoramic Gaussian Scaffold11 Jun
  3. 2026Latent Spatial Memory for Video World Models8 Jun
  4. 2026Prisma-World: Camera-Controllable Multi-Agent Video World Model8 Jun
  5. 2026AnchorWorld: Embodied Egocentric World Simulation with View-based Evolution Customization5 Jun
  6. 2026MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Data1 Jun
  7. 2026Gamma-World: Generative Multi-Agent World Modeling Beyond Two Players27 May
  8. 2026Teaching Video Generators to Remember: Eliciting Dynamic Memory for Out-of-Sight State Evolution25 May
  9. 2026SCOPE: Simulating Cross-game Operations in Playable Environments for FPS World Models22 May
  10. 2026Incantation: Natural Language as the Action Interface for Multi-Entity Video World Models18 May
  11. 2026ChronoAgentic: A Code-based Multi-Agent World Simulator for Physically Grounded Simulation Construction14 May
  12. 2026MultiWorld: Scalable Multi-Agent Multi-View Video World Models20 Apr
  13. 2026Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory10 Apr
  14. 2026Out of Sight but Not Out of Mind: Hybrid Memory for Dynamic Video World Models26 Mar
  15. 2026MosaicMem: Hybrid Spatial Memory for Controllable Video World Models17 Mar
  16. 2026Out of Sight, Out of Mind? Evaluating State Evolution in Video World Models13 Mar
  17. 2026LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Models7 Mar
  18. 2026Beyond Pixel Histories: World Models with Persistent 3D State3 Mar
  19. 2026Solaris: Building a Multiplayer Video World Model in Minecraft25 Feb
  20. 2026Advancing Open-source World Models28 Jan
  21. 2026NeoVerse: Enhancing 4D World Model with in-the-wild Monocular Videos1 Jan
  22. 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling16 Dec
  23. 2025RELIC: Interactive Video World Model with Long-Horizon Memory3 Dec
  24. 2025WorldPack: Dynamic Frame Compression for Long-context Video World Modeling2 Dec
  25. 2025Hunyuan-GameCraft-2: Instruction-following Interactive Game World Model28 Nov
  26. 2025Memory Forcing: Spatio-Temporal Memory for Consistent Scene Generation on Minecraft3 Oct
  27. 2025Matrix-game 2.0: An open-source real-time and streaming interactive world model18 Aug
  28. 2025Yume: An Interactive World Generation Model23 Jul
  29. 2025Video World Models with Long-term Spatial Memory5 Jun
  30. 2025DeepVerse: 4D Autoregressive Video Generation as a World Model1 Jun
  31. 2025VRAG: Learning World Models for Interactive Video Generation28 May
  32. 2025WorldMem: Long-term Consistent World Simulation with Memory16 Apr
  33. 2025MineWorld: a Real-Time and Open-Source Interactive World Model on Minecraft11 Apr
  34. 2025WorldModelBench: Judging Video Generation Models As World Models28 Feb
  35. 2025GameFactory: Creating New Games with Generative Interactive Videos14 Jan
  36. 2024GameGen-X: Interactive Open-world Game Video Generation1 Nov
  37. 2024Diffusion Models Are Real-Time Game Engines27 Aug
  38. 2024Genie: Generative Interactive Environments23 Feb

50 newest of 53 All 53 (JSON)

Other references · 45

  1. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  2. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  3. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  4. 2024Depth Anything V2cited by 1,422
  5. 2025SAM 3: Segment Anything with Conceptscited by 906
  6. 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
  7. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 464
  8. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 403
  9. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 341
  10. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 300
  11. 2024Navigation World Modelscited by 272
  12. 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 202
33 more
  1. 2025World Simulation with Video Foundation Models for Physical AIcited by 200
  2. 2025Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrievalcited by 163
  3. 2024RGBleftrightarrowX: Image decomposition and synthesis using material- and lighting-aware diffusion modelscited by 161
  4. 2023SceneScape: Text-Driven Consistent Scene Generationcited by 160
  5. 2025WorldScore: A Unified Evaluation Benchmark for World Generationcited by 156
  6. 2025VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memorycited by 109
  7. 2024WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environmentcited by 107
  8. 2025Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Controlcited by 106
  9. 2025Voyager: Long-Range and World-Consistent Video Diffusion for Explorable 3D Scene Generationcited by 101
  10. 2025Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Conditioncited by 98
  11. 2025Aether: Geometric-Aware Unified World Modelingcited by 97
  12. 2025Yume-1.5: A Text-Controlled Interactive World Generation Modelcited by 84
  13. 2024The Matrix: Infinite-Horizon World Generation with Real-Time Moving Controlcited by 80
  14. 2026Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learningcited by 66
  15. 2026Infinite Worlds with Versatile Interactionscited by 48
  16. 2026Lyra 2.0: Explorable Generative 3D Worldscited by 44
  17. 2025Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillationcited by 41
  18. 2025Spatia: Video Generation with Updatable Spatial Memorycited by 40
  19. 2025PoE-World: Compositional World Modeling with Products of Programmatic Expertscited by 26
  20. 2025Dexterous World Modelscited by 18
  21. 2026Hand2World: Autoregressive Egocentric Interaction Generation via Free-Space Hand Gesturescited by 16
  22. 2026AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memoriescited by 15
  23. 2026EgoSim: Egocentric World Simulator for Embodied Interaction Generationcited by 14
  24. 2026Generated Reality: Human-centric World Simulation using Interactive Video Generation with Hand and Camera Controlcited by 13
  25. 2026LOME: Learning Human-Object Manipulation with Action-Conditioned Egocentric World Modelcited by 13
  26. 2026Egocentric World Model for Photorealistic Hand-Object Interaction Synthesiscited by 12
  27. 2025From Virtual Games to Real-World Playcited by 9
  28. 2026CineScene: Implicit 3D as Effective Scene Representation for Cinematic Video Generationcited by 7
  29. 2025OmniX: From Unified Panoramic Generation and Perception to Graphics-Ready 3D Scenescited by 4
  30. 2026Magpie: Real-Time World Renderer for Interactive Gamescited by 3
  31. 2026ShadowDancer: Teaching Video World Models Any Action by Learning Unified Dynamics Representations from a Video and Its Shadowcited by 2
  32. 2026Video Models as Native 4D Renderers: World-Grounded Conditioning from Animated Meshcited by 2
  33. 2026IMPACT: Attention Is the Interaction Map for Scalable Interaction-Aware World Model Trainingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Code World Model: Coding Agent as World Brain
  2. 2026Oneira: From Open-Ended Generation to Open-World Interaction in Video World Models
  3. 2026StateCraft: Augmenting Video World Models with Abstract-State Reasoning
  4. 2026VisualPatchWorld: Code World Models as Latent Structured Representations for Planning
  5. 2026ExeWorldBench: Benchmarking Code-Based Executable 3D World Modeling
  6. 2025From Masks to Worlds: A Hitchhiker's Guide to World Models
  7. 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends
  8. 2025World-in-World: World Models in a Closed-Loop World