mlsanityData to 14 Oct 2026

Autoregressive Video Generation without Vector Quantization

arXiv 2412.14169 ↗First posted 18 Dec 2024cs.CVcited by 186

On the map: Diffusion models → Video generation

Abstract (arXiv)

This paper presents a novel approach that enables autoregressive video generation with high efficiency. We propose to reformulate the video generation problem as a non-quantized autoregressive modeling of temporal frame-by-frame prediction and spatial set-by-set prediction. Unlike raster-scan prediction in prior autoregressive models or joint distribution modeling of fixed-length tokens in diffusion models, our approach maintains the causal property of GPT-style models for flexible in-context capabilities, while leveraging bidirectional modeling within individual frames for efficiency. With the proposed approach, we train a novel video autoregressive model without vector quantization, termed NOVA. Our results demonstrate that NOVA surpasses prior autoregressive video models in data efficiency, inference speed, visual fidelity, and video fluency, even with a much smaller model capacity, i.e., 0.6B parameters. NOVA also outperforms state-of-the-art image diffusion models in text-to-image generation tasks, with a significantly lower training cost. Additionally, NOVA generalizes well across extended video durations and enables diverse zero-shot applications in one unified model. Code and models are publicly available at https://github.com/baaivision/NOVA.

Key facts. Cited by 186 papers in the map (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. First posted on arXiv 18 Dec 2024.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 95; 2026: 90.

186 papers.

Sort:

By area:

  1. 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
  2. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 625
  3. 2025LongLive: Real-time Interactive Long Video Generationcited by 265
  4. 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 253
  5. 2025Unified Video Action Modelcited by 205
  6. 2025Self-Forcing++: Towards Minute-Scale High-Quality Video Generationcited by 192
  7. 2026Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generationcited by 184
  8. 2025Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrievalcited by 163
  9. 2025Mogao: An Omni Foundation Model for Interleaved Multi-Modal Generationcited by 109
  10. 2025Echo-4o: Harnessing the Power of GPT-4o Synthetic Images for Improved Image Generationcited by 100
  11. 2025Infinity-RoPE: Action-Controllable Infinite Video Generation Emerges From Autoregressive Self-Rolloutcited by 86
  12. 2025Reward Forcing: Efficient Streaming Video Generation with Rewarded Distribution Matching Distillationcited by 86
38 more
  1. 2025MotionStream: Real-Time Video Generation with Interactive Motion Controlscited by 76
  2. 2026Helios: Real Real-Time Long Video Generation Modelcited by 65
  3. 2025Dual Data Alignment Makes AI-Generated Image Detector Easier Generalizablecited by 60
  4. 2026Context Forcing: Consistent Autoregressive Video Generation with Long Contextcited by 59
  5. 2025GPT-ImgEval: A Comprehensive Benchmark for Diagnosing GPT4o in Image Generationcited by 58
  6. 2025UltraVideo: High-Quality UHD Video Dataset with Comprehensive Captionscited by 57
  7. 2026Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generationcited by 56
  8. 2025End-to-End Training for Autoregressive Video Diffusion via Self-Resamplingcited by 52
  9. 2026World Action Models: The Next Frontier in Embodied AIcited by 50
  10. 2025Evaluating Newtonian Mechanics in Video Generative Models with Real Physical Systemscited by 48
  11. 2025InfinityStar: Unified Spacetime AutoRegressive Modeling for Visual Generationcited by 45
  12. 2025MemFlow: Flowing Adaptive Memory for Consistent and Efficient Long Video Narrativescited by 45
  13. 2025VideoSSM: Autoregressive Long Video Generation with Hybrid State-Space Memorycited by 42
  14. 2025Video-T1: Test-Time Scaling for Video Generationcited by 40
  15. 2026LoL: Longer than Longer, Scaling Video Generation to Hourcited by 32
  16. 2026Rolling Sink: Bridging Limited-Horizon Training and Open-Ended Testing in Autoregressive Video Diffusioncited by 32
  17. 2025Energy-Based Transformers are Scalable Learners and Thinkerscited by 31
  18. 2025SpeakerVid-5M: A Large-Scale High-Quality Dataset for Audio-Visual Dyadic Interactive Human Generationcited by 30
  19. 2026LongLive-2.0: An NVFP4 Parallel Infrastructure for Long Video Generationcited by 30
  20. 2025Neighboring Autoregressive Modeling for Efficient Visual Generationcited by 29
  21. 2025Frequency Autoregressive Image Generation with Continuous Tokenscited by 27
  22. 2026MemRoPE: Training-Free Infinite Video Generation via Evolving Memory Tokenscited by 27
  23. 2024Next Patch Prediction for Autoregressive Visual Generationcited by 26
  24. 2025LMM4LMM: Benchmarking and Evaluating Large-multimodal Image Generation with LMMscited by 24
  25. 2025Geometry-aware 4D Video Generation for Robot Manipulationcited by 24
  26. 2025FlexVAR: Flexible Visual Autoregressive Modeling without Residual Predictioncited by 23
  27. 2025A Survey of Interactive Generative Videocited by 23
  28. 2025Seeing What Matters: Generalizable AI-generated Video Detection with Forensic-Oriented Augmentationcited by 23
  29. 2025Lumos-1: On Autoregressive Video Generation with Discrete Diffusion from a Unified Model Perspectivecited by 22
  30. 2025Macro-from-Micro Planning for High-Quality and Parallelized Autoregressive Long Video Generationcited by 21
  31. 2026INSPATIO-WORLD: A Real-Time 4D World Simulator via Spatiotemporal Autoregressive Modelingcited by 21
  32. 2025MIDAS: Multimodal Interactive Digital-humAn Synthesis via Real-time Autoregressive Video Generationcited by 19
  33. 2025From Pixels to Words -- Towards Native Vision-Language Primitives at Scalecited by 18
  34. 2025FARMER: Flow AutoRegressive Transformer over Pixelscited by 18
  35. 2026Streaming Autoregressive Video Generation via Diagonal Distillationcited by 18
  36. 2025Generative Pre-trained Autoregressive Diffusion Transformercited by 16
  37. 2025VideoMAR: Autoregressive Video Generatio with Continuous Tokenscited by 16
  38. 2025ARIG: Autoregressive Interactive Head Generation for Real-time Conversationscited by 16

50 most cited of 186 All 186 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024ViD-GPT: Introducing GPT-style Autoregressive Generation in Video Diffusion Models
  2. 2025Fast Autoregressive Models for Continuous Latent Generation
  3. 2026UniTemp: Unlocking Video Generation in Any Temporal Order via Bidirectional Distillation
  4. 2024Progressive Autoregressive Video Diffusion Models
  5. 2026VideoAR: Autoregressive Video Generation via Next-Frame & Scale Prediction
  6. 2025Long-Context Autoregressive Video Modeling with Next-Frame Prediction
  7. 2025Playing with Transformer at 30+ FPS via Next-Frame Diffusion
  8. 2024Ca2-VDM: Efficient Autoregressive Video Diffusion Model with Causal Generation and Cache Sharing