mlsanityData to 7 Oct 2026

Scaling Autoregressive Video Models

arXiv 1906.02634 ↗First posted 6 Jun 2019cs.CV cs.AI cs.LGcited by 189

Not on the map. Why?

Cited by 48 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Due to the statistical complexity of video, the high degree of inherent stochasticity, and the sheer amount of data, generating natural video remains a challenging task. State-of-the-art video generation models often attempt to address these issues by combining sometimes complex, usually video-specific neural network architectures, latent variable models, adversarial training and a range of other methods. Despite their often high complexity, these approaches still fall short of generating high quality video continuations outside of narrow domains and often struggle with fidelity. In contrast, we show that conceptually simple autoregressive video generation models based on a three-dimensional self-attention mechanism achieve competitive results across multiple metrics on popular benchmark datasets, for which they produce continuations of high fidelity and realism. We also present results from training our models on Kinetics, a large scale action recognition dataset comprised of YouTube videos exhibiting phenomena such as camera movement, complex object interactions and diverse human movement. While modeling these phenomena consistently remains elusive, we hope that our results, which include occasional realistic continuations encourage further research on comparatively complex, large scale datasets such as Kinetics.

Key facts. Cited by 189 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 30; 2022: 47; 2023: 30; 2024: 30; 2025: 31; 2026: 21.

189 papers.

Sort:

By area:

  1. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  2. 2022Video Diffusion Modelscited by 2,280
  3. 2021ViViT: A Video Vision Transformercited by 1,610
  4. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
  5. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  6. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  7. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
  8. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
  9. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
  10. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 536
  11. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  12. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generationcited by 385
38 more
  1. 2022MAGVIT: Masked Generative Video Transformercited by 341
  2. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Drivingcited by 309
  3. 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolationcited by 307
  4. 2021StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2cited by 300
  5. 2022Flexible Diffusion Modeling of Long Videoscited by 294
  6. 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Drivingcited by 258
  7. 2021N"UWA: Visual Synthesis Pre-training for Neural visUal World creAtioncited by 253
  8. 2022Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformercited by 244
  9. 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Timecited by 209
  10. 2025Unified Video Action Modelcited by 195
  11. 2022Generating Videos with Dynamics-aware Implicit Generative Adversarial Networkscited by 186
  12. 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generationcited by 186
  13. 2021PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learningcited by 181
  14. 2022SimVP: Simpler yet Better Video Predictioncited by 181
  15. 2023Video Probabilistic Diffusion Models in Projected Latent Spacecited by 171
  16. 2021A Good Image Generator Is What You Need for High-Resolution Video Synthesiscited by 162
  17. 2026Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generationcited by 159
  18. 2022Earthformer: Exploring Space-Time Transformers for Earth System Forecastingcited by 146
  19. 2022TransDreamer: Reinforcement Learning with Transformer World Modelscited by 143
  20. 2024Real-Time Video Generation with Pyramid Attention Broadcastcited by 140
  21. 2022Diffusion Models for Video Prediction and Infillingcited by 134
  22. 2021TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?cited by 127
  23. 2021DriveGAN: Towards a Controllable High-Quality Neural Simulationcited by 125
  24. 2022What to Hide from Your Students: Attention-Guided Masked Image Modelingcited by 117
  25. 2024OmniTokenizer: A Joint Image-Video Tokenizer for Visual Generationcited by 117
  26. 2024Generative Camera Dolly: Extreme Monocular Dynamic Novel View Synthesiscited by 116
  27. 2022MaskViT: Masked Visual Pre-Training for Video Predictioncited by 112
  28. 2022Reinforcement Learning with Action-Free Pre-Training from Videoscited by 107
  29. 2024Rolling Diffusion Modelscited by 102
  30. 2023SimDA: Simple Diffusion Adapter for Efficient Video Generationcited by 101
  31. 2023PreDiff: Precipitation Nowcasting with Latent Diffusion Modelscited by 96
  32. 2021Colorization Transformercited by 89
  33. 2021FitVid: Overfitting in Pixel-Level Video Predictioncited by 86
  34. 2022PoseGPT: Quantization-based 3D Human Motion Generation and Forecastingcited by 78
  35. 2022VIDM: Video Implicit Diffusion Modelscited by 78
  36. 2025Infinity-RoPE: Action-Controllable Infinite Video Generation Emerges From Autoregressive Self-Rolloutcited by 78
  37. 2023ARTboldsymbol·V: Auto-Regressive Text-to-Video Generation with Diffusion Modelscited by 74
  38. 2025MotionStream: Real-Time Video Generation with Interactive Motion Controlscited by 71

50 most cited of 189 All 189 (JSON)