# VideoMAR: Autoregressive Video Generatio with Continuous Tokens

arXiv: https://arxiv.org/abs/2506.14168 · HTML: https://mlsanity.com/paper/2506.14168/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 40 references that are in the map.
- First posted on arXiv 17 Jun 2025.

## Builds on (most cited first, 30 of 37)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [MaskGIT: Masked Generative Image Transformer](https://mlsanity.com/paper/2202.04200/) (2022)
- [Cosmos World Foundation Model Platform for Physical AI](https://mlsanity.com/paper/2501.03575/) (2025)
- [Emu3: Next-Token Prediction is All You Need](https://mlsanity.com/paper/2409.18869/) (2024)
- [Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation](https://mlsanity.com/paper/2406.06525/) (2024)
- [Genie: Generative Interactive Environments](https://mlsanity.com/paper/2402.15391/) (2024)
- [Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion](https://mlsanity.com/paper/2407.01392/) (2024)
- [Autoregressive Image Generation without Vector Quantization](https://mlsanity.com/paper/2406.11838/) (2024)
- [VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models](https://mlsanity.com/paper/2401.09047/) (2024)
- [VideoPoet: A Large Language Model for Zero-Shot Video Generation](https://mlsanity.com/paper/2312.14125/) (2023)
- [From Slow Bidirectional to Fast Autoregressive Video Diffusion Models](https://mlsanity.com/paper/2412.07772/) (2024)
- [I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Models](https://mlsanity.com/paper/2311.04145/) (2023)
- [MAGVIT: Masked Generative Video Transformer](https://mlsanity.com/paper/2212.05199/) (2022)
- [MAGI-1: Autoregressive Video Generation at Scale](https://mlsanity.com/paper/2505.13211/) (2025)
- [StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text](https://mlsanity.com/paper/2403.14773/) (2024)
- [SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Prediction](https://mlsanity.com/paper/2310.20700/) (2023)
- [DART: Noise Injection for Robust Imitation Learning](https://mlsanity.com/paper/1703.09327/) (2017)
- [Autoregressive Video Generation without Vector Quantization](https://mlsanity.com/paper/2412.14169/) (2024)
- [Fluid: Scaling Autoregressive Text-to-image Generative Models with Continuous Tokens](https://mlsanity.com/paper/2410.13863/) (2024)
- [ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generation](https://mlsanity.com/paper/2402.04324/) (2024)
- [Bridging the Gap between Training and Inference for Neural Machine Translation](https://mlsanity.com/paper/1906.02448/) (2019)
- [Loong: Generating Minute-level Long Videos with Autoregressive Language Models](https://mlsanity.com/paper/2410.02757/) (2024)

## Built on by (most cited first, 16 of 16)

- [Lumos-1: On Autoregressive Video Generation with Discrete Diffusion from a Unified Model Perspective](https://mlsanity.com/paper/2507.08801/) (2025)
- [Past- and Future-Informed KV Cache Policy with Salience Estimation in Autoregressive Video Diffusion](https://mlsanity.com/paper/2601.21896/) (2026)
- [PackCache: A Training-Free Acceleration Method for Unified Autoregressive Video Generation via Compact KV-Cache](https://mlsanity.com/paper/2601.04359/) (2026)
- [Towards Better & Faster Autoregressive Image Generation: From the Perspective of Entropy](https://mlsanity.com/paper/2510.09012/) (2025)
- [Uniform Discrete Diffusion with Metric Path for Video Generation](https://mlsanity.com/paper/2510.24717/) (2025)
- [Future Forcing: Future-aware Training-free KV Cache Policy for Autoregressive Video Generation](https://mlsanity.com/paper/2605.30083/) (2026)
- [VideoAR: Autoregressive Video Generation via Next-Frame & Scale Prediction](https://mlsanity.com/paper/2601.05966/) (2026)
- [Efficient Video Diffusion Models: Advancements and Challenges](https://mlsanity.com/paper/2604.15911/) (2026)
- [A Mechanistic View on Video Generation as World Models: State and Dynamics](https://mlsanity.com/paper/2601.17067/) (2026)
- [Directing the World: Fast Autoregressive Video Generation with Compositional Human-Camera Control](https://mlsanity.com/paper/2606.27964/) (2026)
- [Stream-T1: Test-Time Scaling for Streaming Video Generation](https://mlsanity.com/paper/2605.04461/) (2026)
- [MAR-GRPO: Stabilized GRPO for AR-diffusion Hybrid Image Generation](https://mlsanity.com/paper/2604.06966/) (2026)
- [RAE-AR: Taming Autoregressive Models with Representation Autoencoders](https://mlsanity.com/paper/2604.01545/) (2026)
- [Beyond Trial-and-Error: Agentic Optimization for Image-to-Video Adherence](https://mlsanity.com/paper/2608.12290/) (2026)
- [CanvasMAR: Improving Masked Autoregressive Video Prediction With Canvas](https://mlsanity.com/paper/2510.13669/) (2025)
- [Bridging Video Understanding and Generation in a Unified Framework](https://mlsanity.com/paper/2606.31326/) (2026)

## Abstract (arXiv)

Masked-based autoregressive models have demonstrated promising image generation capability in continuous space. However, their potential for video generation remains under-explored. In this paper, we propose VideoMAR, a concise and efficient decoder-only autoregressive image-to-video model with continuous tokens, composing temporal frame-by-frame and spatial masked generation. We first identify temporal causality and spatial bi-directionality as the first principle of video AR models, and propose the next-frame diffusion loss for the integration of mask and video generation. Besides, the huge cost and difficulty of long sequence autoregressive modeling is a basic but crucial issue. To this end, we propose the temporal short-to-long curriculum learning and spatial progressive resolution training, and employ progressive temperature strategy at inference time to mitigate the accumulation error. Furthermore, VideoMAR replicates several unique capacities of language models to video generation. It inherently bears high efficiency due to simultaneous temporal-wise KV cache and spatial-wise parallel generation, and presents the capacity of spatial and temporal extrapolation via 3D rotary embeddings. On the VBench-I2V benchmark, VideoMAR surpasses the previous state-of-the-art (Cosmos I2V) while requiring significantly fewer parameters (9.3%), training data (0.5%), and GPU resources (0.2%).