Code2Worlds: Empowering Coding LLMs for 4D World Generation
Not on the map. Why?
Abstract (arXiv)
Achieving spatial intelligence requires moving beyond visual plausibility to build world simulators grounded in physical laws. While coding LLMs have advanced static 3D scene generation, extending this paradigm to 4D dynamics remains a critical frontier. This task presents two fundamental challenges: multi-scale context entanglement, where monolithic generation fails to balance local object structures with global environmental layouts; and a semantic-physical execution gap, where open-loop code generation leads to physical hallucinations lacking dynamic fidelity. We introduce Code2Worlds, a framework that formulates 4D generation as language-to-simulation code generation. First, we propose a dual-stream architecture that disentangles retrieval-augmented object generation from hierarchical environmental orchestration. Second, to ensure dynamic fidelity, we establish a physics-aware closed-loop mechanism in which a PostProcess Agent scripts dynamics, coupled with a VLM-Motion Critic that performs self-reflection to iteratively refine simulation code. Evaluations on the Code4D benchmark show Code2Worlds outperforms baselines with a 41% SGS gain and 49% higher Richness, while uniquely generating physics-aware dynamics absent in prior static methods. Code: https://github.com/AIGeeksGroup/Code2Worlds. Website: https://aigeeksgroup.github.io/Code2Worlds.
Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 23 of its 30 references that are in the map. First posted on arXiv 12 Feb 2026.
Builds on
23 of 30 references.
References · 23
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2023Visual Instruction Tuningcited by 7,792
- 2024GPT-4o System Cardcited by 4,467
- 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
- 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
- 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 551
- 2024Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMscited by 224
- 2023DreamGaussian4D: Generative 4D Gaussian Splattingcited by 191
11 more
- 2023Infinite Photorealistic Worlds using Procedural Generationcited by 142
- 2024Infinigen Indoors: Photorealistic Indoor Scenes using Procedural Generationcited by 129
- 2024SceneCraft: An LLM Agent for Synthesizing 3D Scene as Blender Codecited by 125
- 20233D-GPT: Procedural 3D Modeling with Large Language Modelscited by 91
- 2023MindAgent: Emergent Gaming Interactioncited by 87
- 2022SinFusion: Training Diffusion Models on a Single Image or Videocited by 58
- 2024Grid Diffusion Models for Text-to-Video Generationcited by 21
- 2025MeshCoder: LLM-Powered Structured Mesh Code Generation from Point Cloudscited by 15
- 2025Procedural Generation of Articulated Simulation-Ready Assetscited by 8
- 2025ImmerseGen: Agent-Guided Immersive World Generation with Alpha-Textured Proxiescited by 5
- 2025VULCAN: Tool-Augmented Multi Agents for Iterative 3D Object Arrangementcited by 3
Built on by
Citing papers per year: 2026: 12.
12 papers.
- 2026Code as Agent Harnesscited by 40
- 2026LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Modelscited by 20
- 2026SimWorld Studio: Automatic Environment Generation with Evolving Coding Agent for Embodied Agent Learningcited by 13
- 2026MWM: Mobile World Models for Action-Conditioned Consistent Predictioncited by 7
- 2026One Video, One World: Turning Monocular Video into Physical 4D Scenescited by 2
- 2026UniMesh: Unifying 3D Mesh Understanding and Generationcited by 1
- 2026Agentic Game Development as a Verifiable Trajectory Data Engine for Scaling World Modelscited by 1
- 2026ProcFunc: Function-Oriented Abstractions for Procedural 3D Generation in Pythoncited by 0
- 2026HappyWorld-Benchcited by 0
- 2026WorldAgent: Verification-Guided Agentic Physical World Constructioncited by 0
- 2026WorldWeave: Growing Persistent Geometric Worlds for Video Generationcited by 0
- 2026AgentGarten: Code Worlds for Evolving Agentscited by 0
- 2026AgentGarten: Code Worlds for Evolving Agents8 Oct 2026
- 2026WorldWeave: Growing Persistent Geometric Worlds for Video Generation28 Sep 2026
- 2026WorldAgent: Verification-Guided Agentic Physical World Construction27 Sep 2026
- 2026HappyWorld-Bench21 Sep 2026
- 2026Agentic Game Development as a Verifiable Trajectory Data Engine for Scaling World Models26 Aug 2026
- 2026One Video, One World: Turning Monocular Video into Physical 4D Scenes30 Jun 2026
- 2026Code as Agent Harness18 May 2026
- 2026SimWorld Studio: Automatic Environment Generation with Evolving Coding Agent for Embodied Agent Learning10 May 2026
- 2026ProcFunc: Function-Oriented Abstractions for Procedural 3D Generation in Python29 Apr 2026
- 2026UniMesh: Unifying 3D Mesh Understanding and Generation19 Apr 2026
- 2026MWM: Mobile World Models for Action-Conditioned Consistent Prediction8 Mar 2026
- 2026LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Models7 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2026GS-Agent: Creating 4D Physical Worlds With Generative Simulation
- 2026Code4Scene: Benchmarking Coding Agents for Constructing and Editing 3D Scenes
- 2026ChronoAgentic: A Code-based Multi-Agent World Simulator for Physically Grounded Simulation Construction
- 2026SimWorlds: A Multi-Agent System for Dynamic 3D Scene Creation
- 2026Recursive Code World Models: Building Complex Worlds through Recursive Scene Programs
- 2026Code2World: A GUI World Model via Renderable Code Generation
- 2026Code World Model: Coding Agent as World Brain
- 2025LatticeWorld: A Multimodal Large Language Model-Empowered Framework for Interactive Complex World Generation