# MAGI-1: Autoregressive Video Generation at Scale

arXiv: https://arxiv.org/abs/2505.13211 · HTML: https://mlsanity.com/paper/2505.13211/

## Key facts

- Cited by 276 papers in the map (data through 14 Oct 2026).
- Placed in Video-generation world models and simulators, one of 11 directions in World models.
- Builds on 54 of its 108 references that are in the map.
- First posted on arXiv 19 May 2025.

## Placed in

- World models / [Video-generation world models and simulators](https://mlsanity.com/wm/video-generation-world-models-and-simulators/)

## Builds on (most cited first, 30 of 54)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [Efficient Streaming Language Models with Attention Sinks](https://mlsanity.com/paper/2309.17453/) (2023)
- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024)
- [GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints](https://mlsanity.com/paper/2305.13245/) (2023)
- [ZeRO: Memory Optimizations Toward Training Trillion Parameter Models](https://mlsanity.com/paper/1910.02054/) (2019)
- [Building Normalizing Flows with Stochastic Interpolants](https://mlsanity.com/paper/2209.15571/) (2022)
- [PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis](https://mlsanity.com/paper/2310.00426/) (2023)
- [Deduplicating Training Data Makes Language Models Better](https://mlsanity.com/paper/2107.06499/) (2021)
- [PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel](https://mlsanity.com/paper/2304.11277/) (2023)
- [Movie Gen: A Cast of Media Foundation Models](https://mlsanity.com/paper/2410.13720/) (2024)
- [LTX-Video: Realtime Video Latent Diffusion](https://mlsanity.com/paper/2501.00103/) (2024)
- [Latte: Latent Diffusion Transformer for Video Generation](https://mlsanity.com/paper/2401.03048/) (2024)
- [FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision](https://mlsanity.com/paper/2407.08608/) (2024)
- [VideoPoet: A Large Language Model for Zero-Shot Video Generation](https://mlsanity.com/paper/2312.14125/) (2023)
- [CLEVRER: CoLlision Events for Video REpresentation and Reasoning](https://mlsanity.com/paper/1910.01442/) (2019)
- [Ring Attention with Blockwise Transformers for Near-Infinite Context](https://mlsanity.com/paper/2310.01889/) (2023)
- [From Slow Bidirectional to Fast Autoregressive Video Diffusion Models](https://mlsanity.com/paper/2412.07772/) (2024)
- [One Step Diffusion via Shortcut Models](https://mlsanity.com/paper/2410.12557/) (2024)
- [Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed](https://mlsanity.com/paper/2101.02388/) (2021)

## Built on by (most cited first, 30 of 276)

- [World Action Models are Zero-shot Policies](https://mlsanity.com/paper/2602.15922/) (2026)
- [LongLive: Real-time Interactive Long Video Generation](https://mlsanity.com/paper/2509.22622/) (2025)
- [Rolling Forcing: Autoregressive Long Video Diffusion in Real Time](https://mlsanity.com/paper/2509.25161/) (2025)
- [Self-Forcing++: Towards Minute-Scale High-Quality Video Generation](https://mlsanity.com/paper/2510.02283/) (2025)
- [Matrix-game 2.0: An open-source real-time and streaming interactive world model](https://mlsanity.com/paper/2508.13009/) (2025)
- [Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generation](https://mlsanity.com/paper/2602.02214/) (2026)
- [Advancing Open-source World Models](https://mlsanity.com/paper/2601.20540/) (2026)
- [Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing](https://mlsanity.com/paper/2508.09192/) (2025)
- [Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory](https://mlsanity.com/paper/2604.08995/) (2026)
- [Controllable Video Generation: A Survey](https://mlsanity.com/paper/2507.16869/) (2025)
- [RELIC: Interactive Video World Model with Long-Horizon Memory](https://mlsanity.com/paper/2512.04040/) (2025)
- [Mixture of Contexts for Long Video Generation](https://mlsanity.com/paper/2508.21058/) (2025)
- [HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixels](https://mlsanity.com/paper/2507.21809/) (2025)
- [SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformer](https://mlsanity.com/paper/2509.24695/) (2025)
- [Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generation](https://mlsanity.com/paper/2506.09350/) (2025)
- [Infinity-RoPE: Action-Controllable Infinite Video Generation Emerges From Autoregressive Self-Rollout](https://mlsanity.com/paper/2511.20649/) (2025)
- [Reward Forcing: Efficient Streaming Video Generation with Rewarded Distribution Matching Distillation](https://mlsanity.com/paper/2512.04678/) (2025)
- [Deep Forcing: Training-Free Long Video Generation with Deep Sink and Participative Compression](https://mlsanity.com/paper/2512.05081/) (2025)
- [Helios: Real Real-Time Long Video Generation Model](https://mlsanity.com/paper/2603.04379/) (2026)
- [Context Forcing: Consistent Autoregressive Video Generation with Long Context](https://mlsanity.com/paper/2602.06028/) (2026)
- [UltraVideo: High-Quality UHD Video Dataset with Comprehensive Captions](https://mlsanity.com/paper/2506.13691/) (2025)
- [Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generation](https://mlsanity.com/paper/2605.15141/) (2026)
- [SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer](https://mlsanity.com/paper/2605.15178/) (2026)
- [Memory Forcing: Spatio-Temporal Memory for Consistent Scene Generation on Minecraft](https://mlsanity.com/paper/2510.03198/) (2025)
- [End-to-End Training for Autoregressive Video Diffusion via Self-Resampling](https://mlsanity.com/paper/2512.15702/) (2025)
- [Infinite Worlds with Versatile Interactions](https://mlsanity.com/paper/2607.07534/) (2026)
- [Learning Latent Action World Models In The Wild](https://mlsanity.com/paper/2601.05230/) (2026)
- [MemFlow: Flowing Adaptive Memory for Consistent and Efficient Long Video Narratives](https://mlsanity.com/paper/2512.14699/) (2025)
- [FlashVSR: Towards Real-Time Diffusion-Based Streaming Video Super-Resolution](https://mlsanity.com/paper/2510.12747/) (2025)
- [VideoSSM: Autoregressive Long Video Generation with Hybrid State-Space Memory](https://mlsanity.com/paper/2512.04519/) (2025)

## Abstract (arXiv)

We present MAGI-1, a world model that generates videos by autoregressively predicting a sequence of video chunks, defined as fixed-length segments of consecutive frames. Trained to denoise per-chunk noise that increases monotonically over time, MAGI-1 enables causal temporal modeling and naturally supports streaming generation. It achieves strong performance on image-to-video (I2V) tasks conditioned on text instructions, providing high temporal consistency and scalability, which are made possible by several algorithmic innovations and a dedicated infrastructure stack. MAGI-1 facilitates controllable generation via chunk-wise prompting and supports real-time, memory-efficient deployment by maintaining constant peak inference cost, regardless of video length. The largest variant of MAGI-1 comprises 24 billion parameters and supports context lengths of up to 4 million tokens, demonstrating the scalability and robustness of our approach. The code and models are available at https://github.com/SandAI-org/MAGI-1 and https://github.com/SandAI-org/MagiAttention. The product can be accessed at https://sand.ai.