MultiGen: Level-Design for Editable Multiplayer Worlds in Diffusion Game Engines
On the map: World models → Video-generation world models and simulators
Abstract (arXiv)
Video world models have shown immense promise for interactive simulation and entertainment, but current systems still struggle with two important aspects of interactivity: user control over the environment for reproducible, editable experiences, and shared inference where players hold influence over a common world. To address these limitations, we introduce an explicit external memory into the system, a persistent state operating independent of the model's context window, that is continually updated by user actions and queried throughout the generation roll-out. Unlike conventional diffusion game engines that operate as next-frame predictors, our approach decomposes generation into Memory, Observation, and Dynamics modules. This design gives users direct, editable control over environment structure via an editable memory representation, and it naturally extends to real-time multiplayer rollouts with coherent viewpoints and consistent cross-player interactions.
Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 74 of its 83 references that are in the map. First posted on arXiv 3 Mar 2026.
Builds on
74 of 83 references.
Closest ancestors · 17
Same direction, newest first
- 2026Infinite-World: Scaling Interactive World Models to 1000-Frame Horizons via Pose-Free Hierarchical Memory2 Feb
- 2025BAgger: Backwards Aggregation for Mitigating Drift in Autoregressive Video Diffusion Models12 Dec
- 2025RELIC: Interactive Video World Model with Long-Horizon Memory3 Dec
- 2025Memory Forcing: Spatio-Temporal Memory for Consistent Scene Generation on Minecraft3 Oct
- 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Time29 Sep
- 2025Yume: An Interactive World Generation Model23 Jul
- 2025Video World Models with Long-term Spatial Memory5 Jun
- 2025Wan: Open and Advanced Large-Scale Video Generative Models26 Mar
- 2025GameFactory: Creating New Games with Generative Interactive Videos14 Jan
- 2024GameGen-X: Interactive Open-world Game Video Generation1 Nov
- 2024Diffusion Models Are Real-Time Game Engines27 Aug
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
5 more
- 2024Genie: Generative Interactive Environments23 Feb
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
- 2022Scalable Diffusion Models with Transformers19 Dec
- 2021VideoGPT: Video Generation using VQ-VAE and Transformers20 Apr
- 2020Denoising Diffusion Probabilistic Models19 Jun
Other references · 57
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
- 2022Video Diffusion Modelscited by 2,280
- 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
- 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
- 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
- 2023Mastering Diverse Domains through World Modelscited by 1,193
- 2020Mastering Atari with Discrete World Modelscited by 990
- 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
- 2024Autoregressive Image Generation without Vector Quantizationcited by 644
- 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
38 more
- 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
- 2024LTX-Video: Realtime Video Latent Diffusioncited by 549
- 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 479
- 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
- 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 450
- 2024Pyramidal Flow Matching for Efficient Video Generative Modelingcited by 354
- 2023Photorealistic Video Generation with Diffusion Modelscited by 304
- 2023Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generationcited by 300
- 2019Scaling Autoregressive Video Modelscited by 189
- 2025Self-Forcing++: Towards Minute-Scale High-Quality Video Generationcited by 176
- 2025History-Guided Video Diffusioncited by 165
- 2025Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrievalcited by 157
- 2017Procedural Content Generation via Machine Learning (PCGML)cited by 136
- 2025Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Modelscited by 126
- 2025Training Agents Inside of Scalable World Modelscited by 122
- 2025Test-Time Training Done Rightcited by 121
- 2025Long-Context Autoregressive Video Modeling with Next-Frame Predictioncited by 119
- 2025Long Context Tuning for Video Generationcited by 110
- 2025VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memorycited by 102
- 2024Loong: Generating Minute-level Long Videos with Autoregressive Language Modelscited by 99
- 2016Playing FPS Games with Deep Reinforcement Learningcited by 95
- 2025Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Conditioncited by 92
- 2025Mixture of Contexts for Long Video Generationcited by 90
- 2020Learning to Simulate Dynamic Environments with GameGANcited by 86
- 2018Evolving Mario Levels in the Latent Space of a Deep Convolutional Generative Adversarial Networkcited by 76
- 2023ARTboldsymbol·V: Auto-Regressive Text-to-Video Generation with Diffusion Modelscited by 74
- 2024Ca2-VDM: Efficient Autoregressive Video Diffusion Model with Causal Generation and Cache Sharingcited by 70
- 2023Level Generation Through Large Language Modelscited by 68
- 2023MarioGPT: Open-Ended Text2Level Generation through Large Language Modelscited by 63
- 2018ViZDoom Competitions: Playing Doom from Pixelscited by 50
- 2024MarDini: Masked Autoregressive Diffusion for Video Generation at Scalecited by 49
- 2024ACDiT: Interpolating Autoregressive Conditional Modeling and Diffusion Transformercited by 45
- 2024EnvGen: Generating and Adapting Environments via LLMs for Training Embodied Agentscited by 41
- 2024ARLON: Boosting Diffusion Transformers with Autoregressive Models for Long Video Generationcited by 41
- 2016Super Mario as a String: Platformer Level Generation Via LSTMscited by 40
- 2024Streetscapes: Large-scale Consistent Street View Generation Using Autoregressive Video Diffusioncited by 39
- 2025Spatia: Video Generation with Updatable Spatial Memorycited by 33
- 2025Next Block Prediction: Video Generation via Semi-Autoregressive Modelingcited by 26
50 most cited of 57 All 57 (JSON)
Built on by
Citing papers per year: 2026: 12.
12 papers.
- 2026Incantation: Natural Language as the Action Interface for Multi-Entity Video World Modelscited by 10
- 2026MASS: Multiplayer World Models with Authoritative Shared Statecited by 7
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trendscited by 6
- 2026Streaming Multi-Agent Autoregressive Diffusion Model with World State Registerscited by 5
- 2026ActionParty: Multi-Subject Action Binding in Generative Video Gamescited by 4
- 2026Echo-Memory: A Controlled Study of Memory in Action World Modelscited by 4
- 2026Population-Scalable Multi-Agent World Modelingcited by 2
- 2026AI for Games in the Foundation Model Eracited by 2
- 2026MilliVid: Hierarchical Latents for Long-Range Consistency in Video Generationcited by 1
- 2026ConsistWorld: Evidence Routing for Consistent Multi-Agent World Modelscited by 1
- 2026Multi-Agent Egocentric World Model with Fine-Grained Embodied Interactioncited by 0
- 2026WorldCast: Distributed Multiplayer World Modelscited by 0
- 2026WorldCast: Distributed Multiplayer World Models8 Oct 2026
- 2026Multi-Agent Egocentric World Model with Fine-Grained Embodied Interaction8 Oct 2026
- 2026ConsistWorld: Evidence Routing for Consistent Multi-Agent World Models18 Sep 2026
- 2026AI for Games in the Foundation Model Era15 Sep 2026
- 2026Population-Scalable Multi-Agent World Modeling9 Aug 2026
- 2026MASS: Multiplayer World Models with Authoritative Shared State6 Aug 2026
- 2026Streaming Multi-Agent Autoregressive Diffusion Model with World State Registers23 Jul 2026
- 2026Echo-Memory: A Controlled Study of Memory in Action World Models8 Jun 2026
- 2026MilliVid: Hierarchical Latents for Long-Range Consistency in Video Generation8 Jun 2026
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends31 May 2026
- 2026Incantation: Natural Language as the Action Interface for Multi-Entity Video World Models18 May 2026
- 2026ActionParty: Multi-Subject Action Binding in Generative Video Games2 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2026From Pixels to States: Rethinking Interactive World Models as Game Engines
- 2026Multiplayer Interactive World Models with Representation Autoencoders
- 2025Matrix-game 2.0: An open-source real-time and streaming interactive world model
- 2026Oneira: From Open-Ended Generation to Open-World Interaction in Video World Models
- 2026ActWorld: From Explorable to Interactive World Model via Action-Aware Memory
- 2026MultiWorld: Scalable Multi-Agent Multi-View Video World Models
- 2026GameWAM: A World Action Model for Video Games
- 2026Matrix-Game 3.5: Enhancing Real-Time Streaming Interactive World Models with Patch Memory