mlsanityData to 7 Oct 2026

Hunyuan-GameCraft-2: Instruction-following Interactive Game World Model

arXiv 2511.23429 ↗First posted 28 Nov 2025cs.CVcited by 58

On the map: World models → Video-generation world models and simulators

Abstract (arXiv)

Recent advances in generative world models have enabled remarkable progress in creating open-ended game environments, evolving from static scene synthesis toward dynamic, interactive simulation. However, current approaches remain limited by rigid action schemas and high annotation costs, restricting their ability to model diverse in-game interactions and player-driven dynamics. To address these challenges, we introduce Hunyuan-GameCraft-2, a new paradigm of instruction-driven interaction for generative game world modeling. Instead of relying on fixed keyboard inputs, our model allows users to control game video contents through natural language prompts, keyboard, or mouse signals, enabling flexible and semantically rich interaction within generated worlds. We formally defined the concept of interactive video data and developed an automated process to transform large-scale, unstructured text-video pairs into causally aligned interactive datasets. Built upon a 14B image-to-video Mixture-of-Experts(MoE) foundation model, our model incorporates a text-driven interaction injection mechanism for fine-grained control over camera motion, character behavior, and environment dynamics. We introduce an interaction-focused benchmark, InterBench, to evaluate interaction performance comprehensively. Extensive experiments demonstrate that our model generates temporally coherent and causally grounded interactive game videos that faithfully respond to diverse and free-form user instructions such as "open the door", "draw a torch", or "trigger an explosion".

Key facts. Cited by 58 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 38 of its 60 references that are in the map. First posted on arXiv 28 Nov 2025.

Builds on

38 of 60 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Memory Forcing: Spatio-Temporal Memory for Consistent Scene Generation on Minecraft3 Oct
  2. 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Time29 Sep
  3. 2025Matrix-game 2.0: An open-source real-time and streaming interactive world model18 Aug
  4. 2025WorldMem: Long-term Consistent World Simulation with Memory16 Apr
  5. 2025GameFactory: Creating New Games with Generative Interactive Videos14 Jan
  6. 2024GameGen-X: Interactive Open-world Game Video Generation1 Nov
  7. 2024Diffusion Models Are Real-Time Game Engines27 Aug
  8. 2024StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text21 Mar
  9. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 29

  1. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  2. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  3. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  4. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  5. 2023One-step Diffusion with Distribution Matching Distillationcited by 978
  6. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 666
  7. 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 648
  8. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
  9. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
  10. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 448
  11. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
  12. 2024Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understandingcited by 294
17 more
  1. 2023LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Modelscited by 242
  2. 2025Video models are zero-shot learners and reasonerscited by 208
  3. 2025Self-Forcing++: Towards Minute-Scale High-Quality Video Generationcited by 176
  4. 2025ViPE: Video Pose Engine for 3D Geometric Perceptioncited by 162
  5. 2024SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Accelerationcited by 158
  6. 2023VideoDirectorGPT: Consistent Multi-scene Video Generation via LLM-Guided Planningcited by 138
  7. 2025HunyuanImage 3.0 Technical Reportcited by 138
  8. 2025Long-Context Autoregressive Video Modeling with Next-Frame Predictioncited by 119
  9. 2023VidToMe: Video Token Merging for Zero-Shot Video Editingcited by 111
  10. 2025One-Minute Video Generation with Test-Time Trainingcited by 100
  11. 2025Voyager: Long-Range and World-Consistent Video Diffusion for Explorable 3D Scene Generationcited by 93
  12. 2024The Matrix: Infinite-Horizon World Generation with Real-Time Moving Controlcited by 79
  13. 2023Free-Bloom: Zero-Shot Text-to-Video Generator with LLM Director and LDM Animatorcited by 74
  14. 2025MotionStream: Real-Time Video Generation with Interactive Motion Controlscited by 71
  15. 2025WorldMirror: Universal 3D World Reconstruction with Any-Prior Promptingcited by 60
  16. 2024ViD-GPT: Introducing GPT-style Autoregressive Generation in Video Diffusion Modelscited by 33
  17. 2025RISE-T2V: Rephrasing and Injecting Semantics with LLM for Expansive Text-to-Video Generationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 56.

58 papers.

Sort:

By area:

  1. 2026Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generationcited by 159
  2. 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modelingcited by 152
  3. 2026Advancing Open-source World Modelscited by 144
  4. 2026Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memorycited by 86
  5. 2026WoVR: World Models as Reliable Simulators for Post-Training VLA Policies with RLcited by 46
  6. 2026Lyra 2.0: Explorable Generative 3D Worldscited by 36
  7. 2026WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluationcited by 34
  8. 2026MIND: Benchmarking Memory Consistency and Action Control in World Modelscited by 31
  9. 2026WorldMark: A Unified Benchmark Suite for Interactive Video World Modelscited by 24
  10. 2026WorldCompass: Reinforcement Learning for Long-Horizon World Modelscited by 22
  11. 2026Out of Sight but Not Out of Mind: Hybrid Memory for Dynamic Video World Modelscited by 20
  12. 2026MultiWorld: Scalable Multi-Agent Multi-View Video World Modelscited by 19
38 more
  1. 2026minWM: A Full-Stack Open-Source Framework for Real-Time Interactive Video World Modelscited by 19
  2. 2026Omni-WorldBench: Towards a Comprehensive Interaction-Centric Evaluation for World Modelscited by 17
  3. 2025TeleWorld: Towards Dynamic Multimodal Synthesis with a 4D World Modelcited by 14
  4. 2026Wonder: Video World Model Done Bettercited by 13
  5. 2026WorldCam: Interactive Autoregressive 3D Gaming Worlds with Camera Pose as a Unifying Geometric Representationcited by 12
  6. 2026SCOPE: Simulating Cross-game Operations in Playable Environments for FPS World Modelscited by 12
  7. 2026Grounding World Simulation Models in a Real-World Metropoliscited by 10
  8. 2026Incantation: Natural Language as the Action Interface for Multi-Entity Video World Modelscited by 10
  9. 2026Olaf-World: Orienting Latent Actions for Video World Modelingcited by 8
  10. 2026Prisma-World: Camera-Controllable Multi-Agent Video World Modelcited by 8
  11. 2026SPIRAL: Self-Evolving Action-Conditioned Video Generation via Reflective Planning Agentscited by 5
  12. 2026Video Generation Models as World Models: Efficient Paradigms, Architectures and Algorithmscited by 5
  13. 2026BiWM: Advancing Open-Source Interactive Video World Models with Bidirectional Autoregressioncited by 5
  14. 2026PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectivescited by 5
  15. 2026Marionette: Predicting World States, Rendering Geometry, Painting Appearancecited by 5
  16. 2026MaineCoon: Pursuing A Real-Time Audio-Visual Social World Modelcited by 4
  17. 2026EgoCS-400K: An Egocentric Gameplay Dataset for World Modelscited by 4
  18. 2026AlayaWorld: Long-Horizon and Playable Video World Generationcited by 4
  19. 2026From Pixels to States: Rethinking Interactive World Models as Game Enginescited by 4
  20. 2026H3-World: Turning Language Understanding into World Controlcited by 4
  21. 2026Research on World Models Is Not Merely Injecting World Knowledge into Specific Taskscited by 3
  22. 2026OpenWorldLib: A Unified Codebase and Definition of Advanced World Modelscited by 3
  23. 2026PanoWorld: Real-World Panoramic Generationcited by 3
  24. 2026Zing-0.5: Toward Playable Worlds with Real-Time Joint Action and Text Controlcited by 3
  25. 2026DecMem: Towards Minute-Long Consistent World Generation with Decoupled Memorycited by 2
  26. 2026AnchorWorld: Embodied Egocentric World Simulation with View-based Evolution Customizationcited by 2
  27. 2026MoWorld: A Flash World Modelcited by 2
  28. 2026WanToFight: Real-Time Generative Game Engine for Multi-Player Combat Interactioncited by 2
  29. 2026ShadowDancer: Teaching Video World Models Any Action by Learning Unified Dynamics Representations from a Video and Its Shadowcited by 2
  30. 2026Sekai2: From World Exploration to Interactive World Modelingcited by 2
  31. 2026AI for Games in the Foundation Model Eracited by 2
  32. 2026OnlineWM: Causality-Aware Active Online Learning for Effective World Modelingcited by 2
  33. 2026OptiWorld: Optimal Control for Video World Generation under Physical Constraintscited by 1
  34. 2026From Zero to Hero: Training-Free Custom Concept Spawning in World Modelscited by 1
  35. 2026Can Video World Models Track Unobserved World States?cited by 1
  36. 2026Precise Editing and Flexible Referencing for Interactable Worldscited by 1
  37. 2026PhysMani: Physics-principled 3D World Model for Dynamic Object Manipulationcited by 0
  38. 2026From Generation to Simulation: How Far Are World Models from Being True Simulators?cited by 0

50 most cited of 58 All 58 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026GameWAM: A World Action Model for Video Games
  2. 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends
  3. 2025Matrix-Game: Interactive World Foundation Model
  4. 2026StatePlay: State-Aware Game World Models for Mechanics-Consistent Generation
  5. 2026Programmable World Model
  6. 2026ActionParty: Multi-Subject Action Binding in Generative Video Games
  7. 2026Game2World Engine: Unlocking In-the-Wild Gameplay Videos for World Model Training
  8. 2026WildWorld: A Large-Scale Dataset for Dynamic World Modeling with Actions and Explicit State toward Generative ARPG