mlsanityData to 7 Oct 2026

MAGI-1: Autoregressive Video Generation at Scale

arXiv 2505.13211 ↗First posted 19 May 2025cs.CV cs.AIcited by 262

On the map: World models → Video-generation world models and simulators

Abstract (arXiv)

We present MAGI-1, a world model that generates videos by autoregressively predicting a sequence of video chunks, defined as fixed-length segments of consecutive frames. Trained to denoise per-chunk noise that increases monotonically over time, MAGI-1 enables causal temporal modeling and naturally supports streaming generation. It achieves strong performance on image-to-video (I2V) tasks conditioned on text instructions, providing high temporal consistency and scalability, which are made possible by several algorithmic innovations and a dedicated infrastructure stack. MAGI-1 facilitates controllable generation via chunk-wise prompting and supports real-time, memory-efficient deployment by maintaining constant peak inference cost, regardless of video length. The largest variant of MAGI-1 comprises 24 billion parameters and supports context lengths of up to 4 million tokens, demonstrating the scalability and robustness of our approach. The code and models are available at https://github.com/SandAI-org/MAGI-1 and https://github.com/SandAI-org/MagiAttention. The product can be accessed at https://sand.ai.

Key facts. Cited by 262 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 54 of its 108 references that are in the map. First posted on arXiv 19 May 2025.

Builds on

54 of 108 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Wan: Open and Advanced Large-Scale Video Generative Models26 Mar
  2. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
  3. 2022Flow Matching for Generative Modeling6 Oct
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale22 Oct

Other references · 49

  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  2. 2022Classifier-Free Diffusion Guidancecited by 5,717
  3. 2013Auto-Encoding Variational Bayescited by 3,980
  4. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  5. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  6. 2022Video Diffusion Modelscited by 2,280
  7. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  8. 2020GLU Variants Improve Transformercited by 1,724
  9. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  10. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  11. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
  12. 2022Building Normalizing Flows with Stochastic Interpolantscited by 964
37 more
  1. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  2. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  3. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 641
  4. 2024Movie Gen: A Cast of Media Foundation Modelscited by 614
  5. 2024LTX-Video: Realtime Video Latent Diffusioncited by 549
  6. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 536
  7. 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 507
  8. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 479
  9. 2019CLEVRER: CoLlision Events for Video REpresentation and Reasoningcited by 475
  10. 2023Ring Attention with Blockwise Transformers for Near-Infinite Contextcited by 448
  11. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
  12. 2024One Step Diffusion via Shortcut Modelscited by 373
  13. 2021Knowledge Distillation in Iterative Generative Models for Improved Sampling Speedcited by 344
  14. 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videoscited by 303
  15. 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 286
  16. 2023DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Modelscited by 239
  17. 2024VideoPhy: Evaluating Physical Commonsense for Video Generationcited by 215
  18. 2024How Far is Video Generation from World Model: A Physical Law Perspectivecited by 204
  19. 2024Flex Attention: A Programming Model for Generating Optimized Attention Kernelscited by 180
  20. 2024Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generationcited by 167
  21. 2025Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Modelcited by 163
  22. 2025Do generative video models understand physical principles?cited by 151
  23. 2022Scaling Laws and Interpretability of Learning from Repeated Datacited by 147
  24. 2016Sequence-to-Sequence Learning as Beam-Search Optimizationcited by 126
  25. 2021GSPMD: General and Scalable Parallelization for ML Computation Graphscited by 109
  26. 2024TorchTitan: One-stop PyTorch native solution for production ready LLM pre-trainingcited by 80
  27. 2022Progressive Distillation for Fast Sampling of Diffusion Modelscited by 61
  28. 2024USP: A Unified Sequence Parallelism Approach for Long Context Generative AIcited by 58
  29. 2024ChatQA 2: Bridging the Gap to Proprietary LLMs in Long Context and RAG Capabilitiescited by 52
  30. 2024VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Modelscited by 42
  31. 2024FlexSP: Accelerating Large Language Model Training via Flexible Sequence Parallelismcited by 19
  32. 2024FlashMask: Efficient and Rich Mask Extension of FlashAttentioncited by 18
  33. 2024Redefining Temporal Modeling in Video Diffusion: The Vectorized Timestep Approachcited by 18
  34. 2024LoongTrain: Efficient Training of Long-Sequence LLMs with Head-Context Parallelismcited by 16
  35. 2021OneFlow: Redesign the Distributed Deep Learning Framework from Scratchcited by 15
  36. 2024Enhancing Training Efficiency Using Packing with Flash Attentioncited by 12
  37. 2025ByteScale: Efficient Scaling of LLM Training with a 2048K Context Length on More Than 12,000 GPUscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 94; 2026: 168.

262 papers.

Sort:

By area:

  1. 2026World Action Models are Zero-shot Policiescited by 411
  2. 2025LongLive: Real-time Interactive Long Video Generationcited by 237
  3. 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Timecited by 209
  4. 2025Self-Forcing++: Towards Minute-Scale High-Quality Video Generationcited by 176
  5. 2025Matrix-game 2.0: An open-source real-time and streaming interactive world modelcited by 174
  6. 2026Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generationcited by 159
  7. 2026Advancing Open-source World Modelscited by 144
  8. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcingcited by 111
  9. 2025Controllable Video Generation: A Surveycited by 93
  10. 2025HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixelscited by 91
  11. 2025SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformercited by 91
  12. 2025Mixture of Contexts for Long Video Generationcited by 90
38 more
  1. 2025RELIC: Interactive Video World Model with Long-Horizon Memorycited by 89
  2. 2026Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memorycited by 86
  3. 2025Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generationcited by 84
  4. 2025Infinity-RoPE: Action-Controllable Infinite Video Generation Emerges From Autoregressive Self-Rolloutcited by 78
  5. 2025Reward Forcing: Efficient Streaming Video Generation with Rewarded Distribution Matching Distillationcited by 75
  6. 2025Deep Forcing: Training-Free Long Video Generation with Deep Sink and Participative Compressioncited by 72
  7. 2026Helios: Real Real-Time Long Video Generation Modelcited by 56
  8. 2025UltraVideo: High-Quality UHD Video Dataset with Comprehensive Captionscited by 52
  9. 2026SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformercited by 50
  10. 2026Context Forcing: Consistent Autoregressive Video Generation with Long Contextcited by 47
  11. 2025Memory Forcing: Spatio-Temporal Memory for Consistent Scene Generation on Minecraftcited by 46
  12. 2025End-to-End Training for Autoregressive Video Diffusion via Self-Resamplingcited by 45
  13. 2026Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generationcited by 44
  14. 2026Learning Latent Action World Models In The Wildcited by 42
  15. 2025FlashVSR: Towards Real-Time Diffusion-Based Streaming Video Super-Resolutioncited by 40
  16. 2025MemFlow: Flowing Adaptive Memory for Consistent and Efficient Long Video Narrativescited by 40
  17. 2025VideoSSM: Autoregressive Long Video Generation with Hybrid State-Space Memorycited by 39
  18. 2026Infinite Worlds with Versatile Interactionscited by 37
  19. 2025Spatia: Video Generation with Updatable Spatial Memorycited by 33
  20. 2025ROSE: Remove Objects with Side Effects in Videoscited by 32
  21. 2026Inference-time Physics Alignment of Video Generative Models with Latent World Modelscited by 32
  22. 2025PAI-Bench: A Comprehensive Benchmark For Physical AIcited by 30
  23. 2025Astra: General Interactive World Model with Autoregressive Denoisingcited by 30
  24. 2026LoL: Longer than Longer, Scaling Video Generation to Hourcited by 29
  25. 2026Rolling Sink: Bridging Limited-Horizon Training and Open-Ended Testing in Autoregressive Video Diffusioncited by 28
  26. 2025Omni-Effects: Unified and Spatially-Controllable Visual Effects Generationcited by 27
  27. 2025StreamDiffusionV2: A Streaming System for Dynamic and Interactive Video Generationcited by 27
  28. 2026Transition Matching Distillation for Fast Video Generationcited by 27
  29. 2025Pack and Force Your Memory: Long-form and Consistent Video Generationcited by 26
  30. 2026LongLive-2.0: An NVFP4 Parallel Infrastructure for Long Video Generationcited by 25
  31. 2026AnyFlow: Any-Step Video Diffusion Model with On-Policy Flow Map Distillationcited by 23
  32. 2026Causal-rCM: A Unified Teacher-Forcing and Self-Forcing Open Recipe for Autoregressive Diffusion Distillation in Streaming Video Generation and Interactive World Modelscited by 23
  33. 2026VideoMemory: Toward Consistent Video Generation via Memory Integrationcited by 22
  34. 2026ShotStream: Streaming Multi-Shot Video Generation for Interactive Storytellingcited by 22
  35. 2025NarrLV: Towards a Comprehensive Narrative-Centric Evaluation for Long Video Generationcited by 21
  36. 2025Kling-Avatar: Grounding Multimodal Instructions for Cascaded Long-Duration Avatar Animation Synthesiscited by 20
  37. 2026MemRoPE: Training-Free Infinite Video Generation via Evolving Memory Tokenscited by 20
  38. 2026Relax Forcing: Relaxed KV-Memory for Consistent Long Video Generationcited by 20

50 most cited of 262 All 262 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Towards Chunk-Wise Generation for Long Videos
  2. 2025AR-Diffusion: Asynchronous Video Generation with Auto-Regressive Diffusion
  3. 2026TempAct: Advancing Temporal Plausibility in Autoregressive Video Generation via Planner-Executor RL
  4. 2025Playing with Transformer at 30+ FPS via Next-Frame Diffusion
  5. 2026Rollout-Marginal Distillation for Long-Horizon Autoregressive Video Generation
  6. 2026The Past Frames the Future: Memory for Autoregressive Video Generation
  7. 2025Memorize-and-Generate: Towards Long-Term Consistency in Real-Time Video Generation
  8. 2025Long-Context Autoregressive Video Modeling with Next-Frame Prediction