mlsanityData to 7 Oct 2026

World Craft: Agentic Framework to Create Visualizable Worlds via Text

arXiv 2601.09150 ↗First posted 14 Jan 2026cs.HCcited by 1

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Large Language Models (LLMs) motivate generative agent simulation (e.g., AI Town) to create a ``dynamic world'', holding immense value across entertainment and research. However, for non-experts, especially those without programming skills, it isn't easy to customize a visualizable environment by themselves. In this paper, we introduce World Craft, an agentic world creation framework to create an executable and visualizable AI Town via user textual descriptions. It consists of two main modules, World Scaffold and World Guild. World Scaffold is a structured and concise standardization to develop interactive game scenes, serving as an efficient scaffolding for LLMs to customize an executable AI Town-like environment. World Guild is a multi-agent framework to progressively analyze users' intents from rough descriptions, and synthesizes required structured contents (eg environment layout and assets) for World Scaffold . Moreover, we construct a high-quality error-correction dataset via reverse engineering to enhance spatial knowledge and improve the stability and controllability of layout generation, while reporting multi-dimensional evaluation metrics for further analysis. Extensive experiments demonstrate that our framework significantly outperforms existing commercial code agents (Cursor and Antigravity) and LLMs (Qwen3 and Gemini-3-Pro). in scene construction and narrative intent conveyance, providing a scalable solution for the democratization of environment creation.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 37 of its 62 references that are in the map. First posted on arXiv 14 Jan 2026.

Builds on

37 of 62 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025WorldCraft: Photo-Realistic 3D World Creation and Customization via LLM Agents21 Feb

Other references · 36

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  5. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  6. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  7. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  8. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  9. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  10. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  11. 2023LIMA: Less Is More for Alignmentcited by 1,080
  12. 2023Teaching Large Language Models to Self-Debugcited by 963
24 more
  1. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
  2. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 679
  3. 20203D-FRONT: 3D Furnished Rooms with layOuts and semaNTicscited by 375
  4. 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agentscited by 350
  5. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 341
  6. 2023On the Planning Abilities of Large Language Models : A Critical Investigationcited by 308
  7. 2022Social Simulacra: Creating Populated Prototypes for Social Computing Systemscited by 300
  8. 2023Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memorycited by 288
  9. 2023Holodeck: Language Guided Generation of 3D Embodied AI Environmentscited by 257
  10. 2020Experience Grounds Languagecited by 241
  11. 2021ATISS: Autoregressive Transformers for Indoor Scene Synthesiscited by 195
  12. 2022Omni3D: A Large Benchmark and Model for 3D Object Detection in the Wildcited by 159
  13. 2023OpenAgents: An Open Platform for Language Agents in the Wildcited by 136
  14. 2024SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understandingcited by 130
  15. 2023MindAgent: Emergent Gaming Interactioncited by 87
  16. 2023AgentSims: An Open-Source Sandbox for Large Language Model Evaluationcited by 86
  17. 2022Mind's Eye: Grounded Language Model Reasoning through Simulationcited by 83
  18. 2023Humanoid Agents: Platform for Simulating Human-like Generative Agentscited by 73
  19. 2022HouseDiffusion: Vector Floorplan Generation via a Diffusion Model with Discrete and Continuous Denoisingcited by 72
  20. 2023Tell2Design: A Dataset for Language-Guided Floor Plan Generationcited by 19
  21. 2025FloorplanQA: A Benchmark for Spatial Reasoning in LLMs using Structured Representationscited by 13
  22. 2025TinyTroupe: An LLM-powered Multiagent Persona Simulation Toolkitcited by 7
  23. 2025ORIGAMISPACE: Benchmarking Multimodal LLMs in Multi-Step Spatial Reasoning with Mathematical Constraintscited by 7
  24. 2025Agent-Kernel: A MicroKernel Multi-Agent System Framework for Adaptive Social Simulation Powered by LLMscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2025WorldGen: From Text to Traversable and Interactive 3D Worlds
  2. 2026Code2Games: Enabling Coding Agents for Gaming World Generation
  3. 2025LatticeWorld: A Multimodal Large Language Model-Empowered Framework for Interactive Complex World Generation
  4. 2026WorldClaw: Agentic 3D Open-World Generation at Scale
  5. 2026WorldAgent: Verification-Guided Agentic Physical World Construction
  6. 2024APT: Architectural Planning and Text-to-Blueprint Construction Using Large Language Models for Open-World Agents
  7. 2026Fictional Worldbuilding: Multi-Agent LLM Collaboration with Hierarchical Context Compression and Iterative Review
  8. 2025Creativity in LLM-based Multi-Agent Systems: A Survey