# Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory

arXiv: https://arxiv.org/abs/2604.08995 · HTML: https://mlsanity.com/paper/2604.08995/

## Key facts

- Cited by 100 papers in the map (data through 14 Oct 2026).
- Placed in Video-generation world models and simulators, one of 11 directions in World models.
- Builds on 44 of its 53 references that are in the map.
- First posted on arXiv 10 Apr 2026.

## Placed in

- World models / [Video-generation world models and simulators](https://mlsanity.com/wm/video-generation-world-models-and-simulators/)
- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 30 of 44)

- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024)
- [InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency](https://mlsanity.com/paper/2508.18265/) (2025)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning](https://mlsanity.com/paper/2506.09985/) (2025)
- [Genie: Generative Interactive Environments](https://mlsanity.com/paper/2402.15391/) (2024)
- [Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion](https://mlsanity.com/paper/2407.01392/) (2024)
- [Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion](https://mlsanity.com/paper/2506.08009/) (2025)
- [DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision](https://mlsanity.com/paper/2312.16256/) (2023)
- [World Action Models are Zero-shot Policies](https://mlsanity.com/paper/2602.15922/) (2026)
- [From Slow Bidirectional to Fast Autoregressive Video Diffusion Models](https://mlsanity.com/paper/2412.07772/) (2024)
- [Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planning](https://mlsanity.com/paper/2601.16163/) (2026)
- [MAGI-1: Autoregressive Video Generation at Scale](https://mlsanity.com/paper/2505.13211/) (2025)
- [LTX-2: Efficient Joint Audio-Visual Foundation Model](https://mlsanity.com/paper/2601.03233/) (2026)
- [Self-Forcing++: Towards Minute-Scale High-Quality Video Generation](https://mlsanity.com/paper/2510.02283/) (2025)
- [Matrix-game 2.0: An open-source real-time and streaming interactive world model](https://mlsanity.com/paper/2508.13009/) (2025)
- [Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generation](https://mlsanity.com/paper/2602.02214/) (2026)
- [ViPE: Video Pose Engine for 3D Geometric Perception](https://mlsanity.com/paper/2508.10934/) (2025)
- [WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling](https://mlsanity.com/paper/2512.14614/) (2025)
- [WorldMem: Long-term Consistent World Simulation with Memory](https://mlsanity.com/paper/2504.12369/) (2025)
- [Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrieval](https://mlsanity.com/paper/2506.03141/) (2025)
- [Advancing Open-source World Models](https://mlsanity.com/paper/2601.20540/) (2026)
- [Training Agents Inside of Scalable World Models](https://mlsanity.com/paper/2509.24527/) (2025)
- [DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos](https://mlsanity.com/paper/2602.06949/) (2026)
- [GameFactory: Creating New Games with Generative Interactive Videos](https://mlsanity.com/paper/2501.08325/) (2025)
- [VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memory](https://mlsanity.com/paper/2506.18903/) (2025)
- [RELIC: Interactive Video World Model with Long-Horizon Memory](https://mlsanity.com/paper/2512.04040/) (2025)
- [Mixture of Contexts for Long Video Generation](https://mlsanity.com/paper/2508.21058/) (2025)
- [HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixels](https://mlsanity.com/paper/2507.21809/) (2025)

## Built on by (most cited first, 30 of 100)

- [SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer](https://mlsanity.com/paper/2605.15178/) (2026)
- [Infinite Worlds with Versatile Interactions](https://mlsanity.com/paper/2607.07534/) (2026)
- [DreamX-World 1.0: A General-Purpose Interactive World Model](https://mlsanity.com/paper/2606.16993/) (2026)
- [WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation](https://mlsanity.com/paper/2605.25874/) (2026)
- [WorldMark: A Unified Benchmark Suite for Interactive Video World Models](https://mlsanity.com/paper/2604.21686/) (2026)
- [MBench: A Comprehensive Benchmark on Memory Capability for Video World Models](https://mlsanity.com/paper/2606.00793/) (2026)
- [Self Gradient Forcing: Native Long Video Extrapolation](https://mlsanity.com/paper/2607.20368/) (2026)
- [Wonder: Video World Model Done Better](https://mlsanity.com/paper/2607.26037/) (2026)
- [Addressable Memory for Video World Models](https://mlsanity.com/paper/2608.07408/) (2026)
- [Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond](https://mlsanity.com/paper/2604.22748/) (2026)
- [ActWorld: From Explorable to Interactive World Model via Action-Aware Memory](https://mlsanity.com/paper/2606.17730/) (2026)
- [Alaya-EVOKE: From Linear-Scaling Supervision to Endless World](https://mlsanity.com/paper/2608.13546/) (2026)
- [Code World Model: Coding Agent as World Brain](https://mlsanity.com/paper/2608.25927/) (2026)
- [SCOPE: Simulating Cross-game Operations in Playable Environments for FPS World Models](https://mlsanity.com/paper/2605.23345/) (2026)
- [EchoWM: Open and Enterable Omnimodal World Models](https://mlsanity.com/paper/2608.23189/) (2026)
- [SolarWM: Open Data and Scalable Training for Long-Horizon Video World Models](https://mlsanity.com/paper/2609.02886/) (2026)
- [Prisma-World: Camera-Controllable Multi-Agent Video World Model](https://mlsanity.com/paper/2606.09507/) (2026)
- [WorldRoamBench: An Open-World Benchmark for Long-Horizon Stability of Interactive World Models](https://mlsanity.com/paper/2606.31672/) (2026)
- [ReWorld: An Interactive World Model with Long-Horizon Memory](https://mlsanity.com/paper/2608.23565/) (2026)
- [ReactiveGWM: Steering NPC in Reactive Game World Models](https://mlsanity.com/paper/2605.15256/) (2026)
- [Matrix-Game 3.5: Enhancing Real-Time Streaming Interactive World Models with Patch Memory](https://mlsanity.com/paper/2608.29910/) (2026)
- [Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends](https://mlsanity.com/paper/2606.01164/) (2026)
- [Wan-Streamer v0.1: End-to-end Real-time Interactive Foundation Models](https://mlsanity.com/paper/2606.25041/) (2026)
- [BiWM: Advancing Open-Source Interactive Video World Models with Bidirectional Autoregression](https://mlsanity.com/paper/2606.10135/) (2026)
- [AlayaWorld: Long-Horizon and Playable Video World Generation](https://mlsanity.com/paper/2607.06291/) (2026)
- [StatePlay: State-Aware Game World Models for Mechanics-Consistent Generation](https://mlsanity.com/paper/2607.26754/) (2026)
- [EgoCS-400K: An Egocentric Gameplay Dataset for World Models](https://mlsanity.com/paper/2606.18180/) (2026)
- [Worldscape-MoE: A Unified Mixture-of-Experts World Model for Scalable Heterogeneous Action Control](https://mlsanity.com/paper/2607.03964/) (2026)
- [PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectives](https://mlsanity.com/paper/2608.13552/) (2026)
- [HorizonDrive: Self-Corrective Autoregressive World Model for Long-horizon Driving Simulation](https://mlsanity.com/paper/2605.11596/) (2026)

## Abstract (arXiv)

With the advancement of interactive video generation, diffusion models have increasingly demonstrated their potential as world models. However, existing approaches still struggle to simultaneously achieve memory-enabled long-term temporal consistency and high-resolution real-time generation, limiting their applicability in real-world scenarios. To address this, we present Matrix-Game 3.0, a memory-augmented interactive world model designed for 720p real-time longform video generation. Building upon Matrix-Game 2.0, we introduce systematic improvements across data, model, and inference. First, we develop an upgraded industrial-scale infinite data engine that integrates Unreal Engine-based synthetic data, large-scale automated collection from AAA games, and real-world video augmentation to produce high-quality Video-Pose-Action-Prompt quadruplet data at scale. Second, we propose a training framework for long-horizon consistency: by modeling prediction residuals and re-injecting imperfect generated frames during training, the base model learns self-correction; meanwhile, camera-aware memory retrieval and injection enable the base model to achieve long horizon spatiotemporal consistency. Third, we design a multi-segment autoregressive distillation strategy based on Distribution Matching Distillation (DMD), combined with model quantization and VAE decoder pruning, to achieve efficient real-time inference. Experimental results show that Matrix-Game 3.0 achieves up to 40 FPS real-time generation at 720p resolution with a 5B model, while maintaining stable memory consistency over minute-long sequences. Scaling up to a 2x14B model further improves generation quality, dynamics, and generalization. Our approach provides a practical pathway toward industrial-scale deployable world models.