mlsanityData to 14 Oct 2026

Matrix-game 2.0: An open-source real-time and streaming interactive world model

arXiv 2508.13009 ↗First posted 18 Aug 2025cs.CVcited by 186

On the map: World models → Video-generation world models and simulators

also in 1 more
Abstract (arXiv)

Recent advances in interactive video generations have demonstrated diffusion model's potential as world models by capturing complex physical dynamics and interactive behaviors. However, existing interactive world models depend on bidirectional attention and lengthy inference steps, severely limiting real-time performance. Consequently, they are hard to simulate real-world dynamics, where outcomes must update instantaneously based on historical context and current actions. To address this, we present Matrix-Game 2.0, an interactive world model generates long videos on-the-fly via few-step auto-regressive diffusion. Our framework consists of three key components: (1) A scalable data production pipeline for Unreal Engine and GTA5 environments to effectively produce massive amounts (about 1200 hours) of video data with diverse interaction annotations; (2) An action injection module that enables frame-level mouse and keyboard inputs as interactive conditions; (3) A few-step distillation based on the casual architecture for real-time and streaming video generation. Matrix Game 2.0 can generate high-quality minute-level videos across diverse scenes at an ultra-fast speed of 25 FPS. We open-source our model weights and codebase to advance research in interactive world modeling.

Key facts. Cited by 186 papers in the map (data through 14 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 49 of its 56 references that are in the map. First posted on arXiv 18 Aug 2025.

Builds on

49 of 56 references.

Closest ancestors · 17

Same direction, newest first

  1. 2025Yume: An Interactive World Generation Model23 Jul
  2. 2025DeepVerse: 4D Autoregressive Video Generation as a World Model1 Jun
  3. 2025Long-Context State-Space Video World Models26 May
  4. 2025MAGI-1: Autoregressive Video Generation at Scale19 May
  5. 2025MineWorld: a Real-Time and Open-Source Interactive World Model on Minecraft11 Apr
  6. 2025Wan: Open and Advanced Large-Scale Video Generative Models26 Mar
  7. 2025GameFactory: Creating New Games with Generative Interactive Videos14 Jan
  8. 2024GameGen-X: Interactive Open-world Game Video Generation1 Nov
  9. 2024Diffusion Models Are Real-Time Game Engines27 Aug
  10. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
  11. 2024Genie: Generative Interactive Environments23 Feb
  12. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
5 more
  1. 2023Learning Interactive Real-World Simulators9 Oct
  2. 2022Scalable Diffusion Models with Transformers19 Dec
  3. 2021Learning Transferable Visual Models From Natural Language Supervision26 Feb
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  5. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 32

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  4. 2023Consistency Modelscited by 1,822
  5. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  6. 2023One-step Diffusion with Distribution Matching Distillationcited by 1,019
  7. 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
  8. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 681
  9. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 640
  10. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 637
  11. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 625
  12. 2024LTX-Video: Realtime Video Latent Diffusioncited by 581
20 more
  1. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 580
  2. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 460
  3. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 433
  4. 2024Pyramidal Flow Matching for Efficient Video Generative Modelingcited by 366
  5. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 341
  6. 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 296
  7. 2024EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditionscited by 235
  8. 2023SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Predictioncited by 233
  9. 2025SkyReels-V2: Infinite-length Film Generative Modelcited by 229
  10. 2023FreeNoise: Tuning-Free Longer Video Diffusion via Noise Reschedulingcited by 180
  11. 2023NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generationcited by 171
  12. 2024Direct-a-Video: Customized Video Generation with User-Directed Camera Movement and Object Motioncited by 167
  13. 2025Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Conditioncited by 98
  14. 2025Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generationcited by 90
  15. 2024MovieDreamer: Hierarchical Generation for Coherent Long Visual Sequencecited by 89
  16. 2025Sekai: A Video Dataset towards World Explorationcited by 67
  17. 2022Expanding Small-Scale Datasets with Guided Imaginationcited by 62
  18. 2024MEMO: Memory-Guided Diffusion for Expressive Talking Video Generationcited by 38
  19. 2025Playing with Transformer at 30+ FPS via Next-Frame Diffusioncited by 20
  20. 2023HiPA: Enabling One-Step Text-to-Image Diffusion Models via High-Frequency-Promoting Adaptationcited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 28; 2026: 158.

186 papers.

Sort:

By area:

  1. 2025Self-Forcing++: Towards Minute-Scale High-Quality Video Generationcited by 192
  2. 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modelingcited by 170
  3. 2026Advancing Open-source World Modelscited by 154
  4. 2025Training Agents Inside of Scalable World Modelscited by 136
  5. 2026Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memorycited by 100
  6. 2025RELIC: Interactive Video World Model with Long-Horizon Memorycited by 96
  7. 2026PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulationcited by 82
  8. 2025MotionStream: Real-Time Video Generation with Interactive Motion Controlscited by 76
  9. 2025OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modelingcited by 62
  10. 2025Hunyuan-GameCraft-2: Instruction-following Interactive Game World Modelcited by 61
  11. 2026Motubrain: An Advanced World Action Model for Robot Controlcited by 57
  12. 2025Memory Forcing: Spatio-Temporal Memory for Consistent Scene Generation on Minecraftcited by 52
38 more
  1. 2026WoVR: World Models as Reliable Simulators for Post-Training VLA Policies with RLcited by 50
  2. 2026Infinite Worlds with Versatile Interactionscited by 48
  3. 2025World-in-World: World Models in a Closed-Loop Worldcited by 47
  4. 2026Solaris: Building a Multiplayer Video World Model in Minecraftcited by 47
  5. 2026Lyra 2.0: Explorable Generative 3D Worldscited by 44
  6. 2026Infinite-World: Scaling Interactive World Models to 1000-Frame Horizons via Pose-Free Hierarchical Memorycited by 40
  7. 2026WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluationcited by 39
  8. 2026MIND: Benchmarking Memory Consistency and Action Control in World Modelscited by 35
  9. 2025Astra: General Interactive World Model with Autoregressive Denoisingcited by 33
  10. 2026LoL: Longer than Longer, Scaling Video Generation to Hourcited by 32
  11. 2025StreamDiffusionV2: A Streaming System for Dynamic and Interactive Video Generationcited by 30
  12. 2026VerseCrafter: Dynamic Realistic Video World Model with 4D Geometric Controlcited by 28
  13. 2026LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Modelscited by 28
  14. 2026MosaicMem: Hybrid Spatial Memory for Controllable Video World Modelscited by 27
  15. 2026Out of Sight but Not Out of Mind: Hybrid Memory for Dynamic Video World Modelscited by 26
  16. 2026WorldMark: A Unified Benchmark Suite for Interactive Video World Modelscited by 26
  17. 2026Causal-rCM: A Unified Teacher-Forcing and Self-Forcing Open Recipe for Autoregressive Diffusion Distillation in Streaming Video Generation and Interactive World Modelscited by 25
  18. 2026WorldCompass: Reinforcement Learning for Long-Horizon World Modelscited by 24
  19. 2026AtomVLA: Scalable Post-Training for Robotic Manipulation via Predictive Latent World Modelscited by 23
  20. 2026RealWonder: Real-Time Physical Action-Conditioned Video Generationcited by 22
  21. 2026INSPATIO-WORLD: A Real-Time 4D World Simulator via Spatiotemporal Autoregressive Modelingcited by 21
  22. 2025Memorize-and-Generate: Towards Long-Term Consistency in Real-Time Video Generationcited by 20
  23. 2026minWM: A Full-Stack Open-Source Framework for Real-Time Interactive Video World Modelscited by 19
  24. 2026WorldStereo: Bridging Camera-Guided Video Generation and Scene Reconstruction via 3D Geometric Memoriescited by 18
  25. 2026Omni-WorldBench: Towards a Comprehensive Interaction-Centric Evaluation for World Modelscited by 18
  26. 2026MBench: A Comprehensive Benchmark on Memory Capability for Video World Modelscited by 18
  27. 2025AstraNav-World: World Model for Foresight Control and Consistencycited by 17
  28. 2025A Survey on Cache Methods in Diffusion Models: Toward Efficient Multi-Modal Generationcited by 16
  29. 2026LIVE: Long-horizon Interactive Video World Modelingcited by 16
  30. 2026ABot-World-0: Infinite Interactive World Rollout on a Single Desktop GPUcited by 16
  31. 2025Vidarc: Embodied Video Diffusion Model for Closed-loop Controlcited by 15
  32. 2026AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memoriescited by 15
  33. 2026WorldKV: Efficient World Memory with World Retrieval and Compressioncited by 15
  34. 2026Gamma-World: Generative Multi-Agent World Modeling Beyond Two Playerscited by 15
  35. 2026Wonder: Video World Model Done Bettercited by 15
  36. 2026Addressable Memory for Video World Modelscited by 15
  37. 2026EgoSim: Egocentric World Simulator for Embodied Interaction Generationcited by 14
  38. 2026iWorld-Bench: A Benchmark for Interactive World Models with a Unified Action Generation Frameworkcited by 14

50 most cited of 186 All 186 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024The Matrix: Infinite-Horizon World Generation with Real-Time Moving Control
  2. 2026Waypoint-1.5: A Real-Time Video World Model for Consumer Hardware
  3. 2026Multiplayer Interactive World Models with Representation Autoencoders
  4. 2025Vid2World: Crafting Video Diffusion Models to Interactive World Models
  5. 2026MultiGen: Level-Design for Editable Multiplayer Worlds in Diffusion Game Engines
  6. 2025Matrix-Game: Interactive World Foundation Model
  7. 2026Streaming Multi-Agent Autoregressive Diffusion Model with World State Registers
  8. 2026GameWAM: A World Action Model for Video Games