mlsanityData to 7 Oct 2026

Code World Model: Coding Agent as World Brain

arXiv 2608.25927 ↗First posted 26 Aug 2026cs.CV cs.AI cs.CLcited by 9

On the map: World models → Video-generation world models and simulators

also in 1 more
Abstract (arXiv)

World models aim to simulate how complex environments evolve under actions and events, yet existing video-based world models primarily learn dynamics from visual observations, which reveal outcomes rather than the underlying knowledge, rules, and mechanisms governing world evolution. This makes it difficult to maintain persistent consequences and support coherent, open-ended evolution. We introduce Code World Model, a framework that separates world evolution from visual realization by combining the reasoning and coding capabilities of language models with the generative priors of video models. A coding agent serves as the world brain, reasoning about events and their consequences and generating executable code to maintain persistent world state and perform rule-consistent evolution. To connect executable state with visual generation, we introduce a proxy representation that encodes frame-wise spatiotemporal constraints and is compiled into a proxy video, which conditions a video model to render high-fidelity visual observations. We further develop data pipelines for constructing aligned proxy-observation pairs from gameplay and real-world videos. After fine-tuning on paired gameplay data, MiniMax-H3 follows proxy-based spatiotemporal specifications from simple interactive worlds built by the coding agent while preserving rich visual details and dynamics. These results demonstrate the potential of combining code for persistent world evolution with video models for flexible visual realization, providing a new path toward open-ended world models.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 85 of its 91 references that are in the map. First posted on arXiv 26 Aug 2026.

Builds on

85 of 91 references.

Closest ancestors · 25

Same direction, newest first

  1. 2026WorldDirector: Building Controllable World Simulators with Persistent Dynamic Memory2 Jul
  2. 2026ActWorld: From Explorable to Interactive World Model via Action-Aware Memory16 Jun
  3. 2026DreamX-World 1.0: A General-Purpose Interactive World Model15 Jun
  4. 2026minWM: A Full-Stack Open-Source Framework for Real-Time Interactive Video World Models28 May
  5. 2026HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worlds15 Apr
  6. 2026Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory10 Apr
  7. 2026MosaicMem: Hybrid Spatial Memory for Controllable Video World Models17 Mar
  8. 2026Infinite-World: Scaling Interactive World Models to 1000-Frame Horizons via Pose-Free Hierarchical Memory2 Feb
  9. 2026Advancing Open-source World Models28 Jan
  10. 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling16 Dec
  11. 2025RELIC: Interactive Video World Model with Long-Horizon Memory3 Dec
  12. 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Time29 Sep
13 more
  1. 2025Matrix-game 2.0: An open-source real-time and streaming interactive world model18 Aug
  2. 2025VRAG: Learning World Models for Interactive Video Generation28 May
  3. 2025Vid2World: Crafting Video Diffusion Models to Interactive World Models20 May
  4. 2025WorldMem: Long-term Consistent World Simulation with Memory16 Apr
  5. 2025MineWorld: a Real-Time and Open-Source Interactive World Model on Minecraft11 Apr
  6. 2025Wan: Open and Advanced Large-Scale Video Generative Models26 Mar
  7. 2025GameFactory: Creating New Games with Generative Interactive Videos14 Jan
  8. 2024GameGen-X: Interactive Open-world Game Video Generation1 Nov
  9. 2024Diffusion Models Are Real-Time Game Engines27 Aug
  10. 2024Pandora: Towards General World Model with Natural Language Actions and Video States12 Jun
  11. 2024Genie: Generative Interactive Environments23 Feb
  12. 2023Learning Interactive Real-World Simulators9 Oct
  13. 2022Scalable Diffusion Models with Transformers19 Dec

Other references · 60

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  8. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  9. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  10. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
  11. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  12. 2018World Modelscited by 1,444
38 more
  1. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  2. 2023Mastering Diverse Domains through World Modelscited by 1,193
  3. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 877
  4. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
  5. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 570
  6. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
  7. 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 507
  8. 2022Transformers are Sample-Efficient World Modelscited by 317
  9. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 285
  10. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  11. 2025LongLive: Real-time Interactive Long Video Generationcited by 237
  12. 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 197
  13. 2025DreamGen: Unlocking Generalization in Robot Learning through Video World Modelscited by 168
  14. 2026Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generationcited by 159
  15. 2025Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrievalcited by 157
  16. 2024iVideoGPT: Interactive VideoGPTs are Scalable World Modelscited by 136
  17. 2025AdaWorld: Learning Adaptable World Models with Latent Actionscited by 108
  18. 2025VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memorycited by 102
  19. 2024WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environmentcited by 99
  20. 2025Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Conditioncited by 92
  21. 2025HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixelscited by 91
  22. 2025CWM: An Open-Weights LLM for Research on Code Generation with World Modelscited by 87
  23. 2020Learning to Simulate Dynamic Environments with GameGANcited by 86
  24. 2025Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generationcited by 84
  25. 2024The Matrix: Infinite-Horizon World Generation with Real-Time Moving Controlcited by 79
  26. 2025Yume-1.5: A Text-Controlled Interactive World Generation Modelcited by 77
  27. 2021Playable Video Generationcited by 63
  28. 2025Pre-Trained Video Generative Models as World Simulatorscited by 43
  29. 2024Generating Code World Models with Large Language Models Guided by Monte Carlo Tree Searchcited by 38
  30. 2026Infinite Worlds with Versatile Interactionscited by 37
  31. 2025PAN: A World Model for General, Actionable, and Long-Horizon World Simulationcited by 36
  32. 2025Matrix-3D: Omnidirectional Explorable 3D World Generationcited by 33
  33. 2025FlashWorld: High-quality 3D Scene Generation within Secondscited by 30
  34. 2022Playable Environments: Video Manipulation in Space and Timecited by 29
  35. 2025WorldExplorer: Towards Generating Fully Navigable 3D Scenescited by 26
  36. 2025Code World Models for General Game Playingcited by 25
  37. 2026World-Gymnast: Training Robots with Reinforcement Learning in a World Modelcited by 25
  38. 2023Promptable Game Models: Text-Guided Game Simulation via Masked Diffusion Modelscited by 22

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 9.

9 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Code Plans, Diffusion Renders: Open-Ended Generative World Modeling
  2. 2026Programmable World Model
  3. 2026VisualPatchWorld: Code World Models as Latent Structured Representations for Planning
  4. 2026PhiZero: A World Model Built Around Physical Language
  5. 2026Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoning
  6. 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends
  7. 2026From Generative Engines to Actionable Simulators: The Imperative of Physical Grounding in World Models
  8. 2025From Masks to Worlds: A Hitchhiker's Guide to World Models