World Agent: Can Language Models Keep a World Running?
On the map: World models → World models for language agents
also in 1 more
Abstract (arXiv)
World models are moving from generating realistic frames to generating playable worlds, yet whether a delivered world can keep running is not tested anywhere. Existing evaluations stop at generation, at delivery, or at single-step transitions, and each stops at a different point along the way. Correct local state transitions or intermediate outcomes do not guarantee a correctly organized causal event flow. We propose the world agent task, which moves the evaluation point of world generation from the moment of delivery to the continued operation that follows. In this task, a model is not asked to generate a world. It is held responsible for keeping the world running, which requires coordinating events and carrying forward their consequences to constrain subsequent evolution. We instantiate the task in WorldAgent-Benchmark with two complementary tracks. In the maintenance track, the model must ground the events of a continuous narrative into correct transitions of the explicit world state while respecting causal, temporal, and concurrency constraints. In the deduction track, the model must predict how the world will evolve under partial observations and act toward a goal. The maintenance track combines LLM-assisted semantic judgments with programmatic validation and scoring, while the deduction track is evaluated entirely programmatically. Individual judgments are auditable against world states and execution logs, and scores can be recomputed from the saved judgments and execution records. Across 8 models, scores decline steadily as pre-built structure is removed from the world, and causal-relation checking is the weakest component for every model. The benchmark makes the continued operation of a world measurable and distinguishes local completion from failures in event organization. Code and dataset will be released on https://github.com/HCPLab-SYSU/WorldAgent-Benchmark.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 34 of its 37 references that are in the map. First posted on arXiv 26 Sep 2026.
Builds on
34 of 37 references.
Closest ancestors · 4
Same direction, newest first
- 2026AI for Games in the Foundation Model Era15 Sep
- 2026PhyAgentOS: A Self-Evolving Operating System for Embodied Agents with Decoupled Cognitive Planning and Physical Execution18 Jul
- 2025Text2World: Benchmarking Large Language Models for Symbolic World Model Generation18 Feb
- 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learning8 Oct
Other references · 30
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
- 2018World Modelscited by 1,444
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
- 2024Genie: Generative Interactive Environmentscited by 706
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 580
- 2015Towards AI-Complete Question Answering: A Set of Prerequisite Toy Taskscited by 309
- 2018TextWorld: A Learning Environment for Text-based Gamescited by 304
- 2024Diffusion Models Are Real-Time Game Enginescited by 282
18 more
- 2021Benchmarking the Spectrum of Agent Capabilitiescited by 193
- 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 175
- 2019Interactive Fiction Games: A Colossal Adventurecited by 165
- 2025WorldMem: Long-term Consistent World Simulation with Memorycited by 152
- 2019Learning to Speak and Act in a Fantasy Text Adventure Gamecited by 122
- 2023Evaluating Cognitive Maps and Planning in Large Language Models with CogEvalcited by 77
- 2018Tracking State Changes in Procedural Text: A Challenge Dataset and Models for Process Paragraph Comprehensioncited by 62
- 2024Can Language Models Serve as Text-Based World Simulators?cited by 50
- 2024Game Generation via Large Language Modelscited by 24
- 2024GAVEL: Generating Games Via Evolution and Language Modelscited by 24
- 2026GameCraft-Bench: Can Agents Build Playable Games End-to-End in a Real Game Engine?cited by 10
- 2025V-GameGym: Visual Game Generation for Code Large Language Modelscited by 8
- 2026WorldRoamBench: An Open-World Benchmark for Long-Horizon Stability of Interactive World Modelscited by 8
- 2026Programmable World Modelcited by 7
- 2025What-If Analysis of Large Language Models: Explore the Game World Using Proactive Thinkingcited by 3
- 2026GameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertionscited by 2
- 2026The GEST-Engine: From Event Graphs to Synthetic Video. A Full Technical Reportcited by 1
- 2026When Stories Evolve: Benchmarking LLM Storytelling Across Agent Architectures in Open-Ended World Simulationscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Qwen-AgentWorld: Language World Models for General Agents
- 2026Why We Need World Models for AGI: Where LLMs Fail and How World Models May Outperform
- 2024Making Large Language Models into World Models with Precondition and Effect Knowledge
- 2026Agent vs. Parametric World Models: Hybrid Planning for Reliable Language Agents
- 2026Bridging the Agent-World Gap: Text World Models for LLM-based Agents
- 2025Web World Models
- 2024Evaluating the World Model Implicit in a Generative Model
- 2025From Word to World: Can Large Language Models be Implicit Text-based World Models?