WorldMind: Decoupled Game World Model for State-Aware NPC Behavior
On the map: World models → Video-generation world models and simulators
Abstract (arXiv)
Game world models have recently demonstrated promising capabilities in generating visually coherent and action-controllable gameplay videos. However, non-player character (NPC) behavior in existing models is either implicitly entangled with video generation or explicitly prescribed through external control signals. Consequently, a game world model has to jointly understand the state, plan the NPC's response and render its visual outcome, limiting its ability to produce responsive and state-aware NPC behavior. The challenge lies in the lack of an explicit interface for state-grounded decision-making. To this end, we introduce WorldMind, to our knowledge the first decoupled framework for state-aware NPC behavior in game world models. WorldMind separates interactive world modeling into four layers: an Understanding Layer that constructs a compact state from generated frames; a Decision Layer that reasons over the compact state to plan the NPC's next action; a Control Layer that translates the actions into temporally aligned conditions; and a Generation Layer that synthesizes their visual outcomes. By reconnecting layers in a closed interaction loop, WorldMind grounds NPC behavior in the evolving game state. We further introduce BOSS-140K, a dataset of gameplay videos paired with rich internal game states, together with an agent that automates the collection at scale. Experiments on BOSS-140K demonstrate reliable compact state reconstruction and mechanics-grounded planning, with WorldMind preferred over the baselines in approximately 70% of pairwise comparisons for its more tactically appropriate and coherent NPC behavior. Project page: https://teawhite.cn/worldmind_projectpage/
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 30 of its 35 references that are in the map. First posted on arXiv 18 Aug 2026.
Builds on
30 of 35 references.
Closest ancestors · 10
Same direction, newest first
- 2026EgoCS-400K: An Egocentric Gameplay Dataset for World Models16 Jun
- 2026SCOPE: Simulating Cross-game Operations in Playable Environments for FPS World Models22 May
- 2026Incantation: Natural Language as the Action Interface for Multi-Entity Video World Models18 May
- 2026ReactiveGWM: Steering NPC in Reactive Game World Models14 May
- 2026WildWorld: A Large-Scale Dataset for Dynamic World Modeling with Actions and Explicit State toward Generative ARPG24 Mar
- 2025Matrix-game 2.0: An open-source real-time and streaming interactive world model18 Aug
- 2025Wan: Open and Advanced Large-Scale Video Generative Models26 Mar
- 2024Diffusion Models Are Real-Time Game Engines27 Aug
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
Other references · 20
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
- 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
- 2020Mastering Atari with Discrete World Modelscited by 990
- 2023One-step Diffusion with Distribution Matching Distillationcited by 978
- 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
- 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 648
- 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
- 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
- 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 338
- 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 285
8 more
- 2019MineRL: A Large-Scale Dataset of Minecraft Demonstrationscited by 210
- 2026Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generationcited by 159
- 2024A Survey on Large Language Model-Based Game Agentscited by 106
- 2017Behavior Trees in Robotics and AI: An Introductioncited by 105
- 2020A Survey of Behavior Trees in Robotics and AIcited by 53
- 2026Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generationcited by 44
- 2026NitroGen: An Open Foundation Model for Generalist Gaming Agentscited by 22
- 2026Scaling Behavior Cloning Improves Causal Reasoning: An Open Model for Real-Time Video Game Playingcited by 10
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2026StatePlay: State-Aware Game World Models for Mechanics-Consistent Generation
- 2026GameWAM: A World Action Model for Video Games
- 2026From Pixels to States: Rethinking Interactive World Models as Game Engines
- 2026Mind-Studio: Executable World Models with Lookahead Evaluation for Partially Observable Games
- 2025Hunyuan-GameCraft-2: Instruction-following Interactive Game World Model
- 2026Code World Model: Coding Agent as World Brain
- 2026Programmable World Model
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends