Code World Model: Coding Agent as World Brain
On the map: World models → Video-generation world models and simulators
also in 1 more
Abstract (arXiv)
World models aim to simulate how complex environments evolve under actions and events, yet existing video-based world models primarily learn dynamics from visual observations, which reveal outcomes rather than the underlying knowledge, rules, and mechanisms governing world evolution. This makes it difficult to maintain persistent consequences and support coherent, open-ended evolution. We introduce Code World Model, a framework that separates world evolution from visual realization by combining the reasoning and coding capabilities of language models with the generative priors of video models. A coding agent serves as the world brain, reasoning about events and their consequences and generating executable code to maintain persistent world state and perform rule-consistent evolution. To connect executable state with visual generation, we introduce a proxy representation that encodes frame-wise spatiotemporal constraints and is compiled into a proxy video, which conditions a video model to render high-fidelity visual observations. We further develop data pipelines for constructing aligned proxy-observation pairs from gameplay and real-world videos. After fine-tuning on paired gameplay data, MiniMax-H3 follows proxy-based spatiotemporal specifications from simple interactive worlds built by the coding agent while preserving rich visual details and dynamics. These results demonstrate the potential of combining code for persistent world evolution with video models for flexible visual realization, providing a new path toward open-ended world models.
Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 85 of its 91 references that are in the map. First posted on arXiv 26 Aug 2026.
Builds on
85 of 91 references.
Closest ancestors · 25
Same direction, newest first
- 2026WorldDirector: Building Controllable World Simulators with Persistent Dynamic Memory2 Jul
- 2026ActWorld: From Explorable to Interactive World Model via Action-Aware Memory16 Jun
- 2026DreamX-World 1.0: A General-Purpose Interactive World Model15 Jun
- 2026minWM: A Full-Stack Open-Source Framework for Real-Time Interactive Video World Models28 May
- 2026HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worlds15 Apr
- 2026Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory10 Apr
- 2026MosaicMem: Hybrid Spatial Memory for Controllable Video World Models17 Mar
- 2026Infinite-World: Scaling Interactive World Models to 1000-Frame Horizons via Pose-Free Hierarchical Memory2 Feb
- 2026Advancing Open-source World Models28 Jan
- 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling16 Dec
- 2025RELIC: Interactive Video World Model with Long-Horizon Memory3 Dec
- 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Time29 Sep
13 more
- 2025Matrix-game 2.0: An open-source real-time and streaming interactive world model18 Aug
- 2025VRAG: Learning World Models for Interactive Video Generation28 May
- 2025Vid2World: Crafting Video Diffusion Models to Interactive World Models20 May
- 2025WorldMem: Long-term Consistent World Simulation with Memory16 Apr
- 2025MineWorld: a Real-Time and Open-Source Interactive World Model on Minecraft11 Apr
- 2025Wan: Open and Advanced Large-Scale Video Generative Models26 Mar
- 2025GameFactory: Creating New Games with Generative Interactive Videos14 Jan
- 2024GameGen-X: Interactive Open-world Game Video Generation1 Nov
- 2024Diffusion Models Are Real-Time Game Engines27 Aug
- 2024Pandora: Towards General World Model with Natural Language Actions and Video States12 Jun
- 2024Genie: Generative Interactive Environments23 Feb
- 2023Learning Interactive Real-World Simulators9 Oct
- 2022Scalable Diffusion Models with Transformers19 Dec
Other references · 60
- 2017Attention Is All You Needcited by 59,932
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2022Training Compute-Optimal Large Language Modelscited by 2,927
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
- 2018World Modelscited by 1,444
38 more
- 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
- 2023Mastering Diverse Domains through World Modelscited by 1,193
- 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 877
- 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
- 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 570
- 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
- 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 507
- 2022Transformers are Sample-Efficient World Modelscited by 317
- 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 285
- 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
- 2025LongLive: Real-time Interactive Long Video Generationcited by 237
- 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 197
- 2025DreamGen: Unlocking Generalization in Robot Learning through Video World Modelscited by 168
- 2026Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generationcited by 159
- 2025Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrievalcited by 157
- 2024iVideoGPT: Interactive VideoGPTs are Scalable World Modelscited by 136
- 2025AdaWorld: Learning Adaptable World Models with Latent Actionscited by 108
- 2025VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memorycited by 102
- 2024WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environmentcited by 99
- 2025Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Conditioncited by 92
- 2025HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixelscited by 91
- 2025CWM: An Open-Weights LLM for Research on Code Generation with World Modelscited by 87
- 2020Learning to Simulate Dynamic Environments with GameGANcited by 86
- 2025Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generationcited by 84
- 2024The Matrix: Infinite-Horizon World Generation with Real-Time Moving Controlcited by 79
- 2025Yume-1.5: A Text-Controlled Interactive World Generation Modelcited by 77
- 2021Playable Video Generationcited by 63
- 2025Pre-Trained Video Generative Models as World Simulatorscited by 43
- 2024Generating Code World Models with Large Language Models Guided by Monte Carlo Tree Searchcited by 38
- 2026Infinite Worlds with Versatile Interactionscited by 37
- 2025PAN: A World Model for General, Actionable, and Long-Horizon World Simulationcited by 36
- 2025Matrix-3D: Omnidirectional Explorable 3D World Generationcited by 33
- 2025FlashWorld: High-quality 3D Scene Generation within Secondscited by 30
- 2022Playable Environments: Video Manipulation in Space and Timecited by 29
- 2025WorldExplorer: Towards Generating Fully Navigable 3D Scenescited by 26
- 2025Code World Models for General Game Playingcited by 25
- 2026World-Gymnast: Training Robots with Reinforcement Learning in a World Modelcited by 25
- 2023Promptable Game Models: Text-Guided Game Simulation via Masked Diffusion Modelscited by 22
50 most cited of 60 All 60 (JSON)
Built on by
Citing papers per year: 2026: 9.
9 papers.
- 2026LynnReal-Omni: Native multi-modal Video Generation for Agentic Visual Workflowscited by 1
- 2026GameDirector: Decoupling Gameplay Logic from Rendering for Player-Configurable Game World Modelscited by 1
- 2026Oneira: From Open-Ended Generation to Open-World Interaction in Video World Modelscited by 1
- 2026Recursive Code World Models: Building Complex Worlds through Recursive Scene Programscited by 0
- 2026Proxy2World: Learning to Generate Worlds From Lightweight Proxies without Seeing Themcited by 0
- 2026Self-Evolving Coding Agents: From Digital Programs to Physical-World Intelligencecited by 0
- 2026LongLive-Plug: Once-for-All Distillation for Video Generationcited by 0
- 2026ROWBench: Do Video Models Render What the Program Specifies?cited by 0
- 2026AgentGarten: Code Worlds for Evolving Agentscited by 0
- 2026AgentGarten: Code Worlds for Evolving Agents8 Oct 2026
- 2026ROWBench: Do Video Models Render What the Program Specifies?1 Oct 2026
- 2026Oneira: From Open-Ended Generation to Open-World Interaction in Video World Models1 Oct 2026
- 2026LongLive-Plug: Once-for-All Distillation for Video Generation29 Sep 2026
- 2026Self-Evolving Coding Agents: From Digital Programs to Physical-World Intelligence28 Sep 2026
- 2026Proxy2World: Learning to Generate Worlds From Lightweight Proxies without Seeing Them28 Sep 2026
- 2026GameDirector: Decoupling Gameplay Logic from Rendering for Player-Configurable Game World Models22 Sep 2026
- 2026LynnReal-Omni: Native multi-modal Video Generation for Agentic Visual Workflows14 Sep 2026
- 2026Recursive Code World Models: Building Complex Worlds through Recursive Scene Programs10 Sep 2026
Similar, not cited
Close in wording, no citation either way
- 2026Code Plans, Diffusion Renders: Open-Ended Generative World Modeling
- 2026Programmable World Model
- 2026VisualPatchWorld: Code World Models as Latent Structured Representations for Planning
- 2026PhiZero: A World Model Built Around Physical Language
- 2026Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoning
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends
- 2026From Generative Engines to Actionable Simulators: The Imperative of Physical Grounding in World Models
- 2025From Masks to Worlds: A Hitchhiker's Guide to World Models