mlsanityData to 7 Oct 2026

Loong: Generating Minute-level Long Videos with Autoregressive Language Models

arXiv 2410.02757 ↗First posted 3 Oct 2024cs.CVcited by 99

Not on the map. Why?

Cited by 34 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

It is desirable but challenging to generate content-rich long videos in the scale of minutes. Autoregressive large language models (LLMs) have achieved great success in generating coherent and long sequences of tokens in the domain of natural language processing, while the exploration of autoregressive LLMs for video generation is limited to generating short videos of several seconds. In this work, we conduct a deep analysis of the challenges that prevent autoregressive LLM-based video generators from generating long videos. Based on the observations and analysis, we propose Loong, a new autoregressive LLM-based video generator that can generate minute-long videos. Specifically, we model the text tokens and video tokens as a unified sequence for autoregressive LLMs and train the model from scratch. We propose progressive short-to-long training with a loss re-weighting scheme to mitigate the loss imbalance problem for long video training. We further investigate inference strategies, including video token re-encoding and sampling strategies, to diminish error accumulation during inference. Our proposed Loong can be trained on 10-second videos and be extended to generate minute-level long videos conditioned on text prompts, as demonstrated by the results. More samples are available at: https://yuqingwang1029.github.io/Loong-video.

Key facts. Cited by 99 papers in the map (data through 7 Oct 2026). Builds on 58 of its 64 references that are in the map. First posted on arXiv 3 Oct 2024.

Builds on

58 of 64 references.

References · 58

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  8. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  9. 2022Scalable Diffusion Models with Transformerscited by 5,931
  10. 2022Generative Adversarial Networkscited by 5,603
  11. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  12. 2017Neural Discrete Representation Learningcited by 4,650
38 more
  1. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  2. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  3. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  4. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  5. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  6. 2022Video Diffusion Modelscited by 2,280
  7. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  8. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  9. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  10. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  11. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  12. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  13. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  14. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
  15. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  16. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  17. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  18. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
  19. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
  20. 2023ModelScope Text-to-Video Technical Reportcited by 651
  21. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  22. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 633
  23. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
  24. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 589
  25. 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 523
  26. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 479
  27. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  28. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 463
  29. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 450
  30. 2016Generating Videos with Scene Dynamicscited by 376
  31. 2022MAGVIT: Masked Generative Video Transformercited by 341
  32. 2023Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generationcited by 300
  33. 2021GODIVA: Generating Open-DomaIn Videos from nAtural Descriptionscited by 254
  34. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 254
  35. 2022Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformercited by 244
  36. 2024StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Textcited by 242
  37. 2023FreeNoise: Tuning-Free Longer Video Diffusion via Noise Reschedulingcited by 173
  38. 2023NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generationcited by 170

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 16; 2025: 48; 2026: 35.

99 papers.

Sort:

By area:

  1. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
  2. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
  3. 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 297
  4. 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Timecited by 209
  5. 2025Video World Models with Long-term Spatial Memorycited by 124
  6. 2025Epona: Autoregressive Diffusion World Model for Autonomous Drivingcited by 115
  7. 2024DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformerscited by 96
  8. 2025MotionStream: Real-Time Video Generation with Interactive Motion Controlscited by 71
  9. 2025RIFLEx: A Free Lunch for Length Extrapolation in Video Diffusion Transformerscited by 63
  10. 2024Liquid: Language Models are Scalable and Unified Multi-modal Generatorscited by 62
  11. 2025Long-Context State-Space Video World Modelscited by 59
  12. 2024Parallelized Autoregressive Visual Generationcited by 58
38 more
  1. 2025UltraVideo: High-Quality UHD Video Dataset with Comprehensive Captionscited by 52
  2. 2025GoT-R1: Unleashing Reasoning Capability of MLLM for Visual Generation with Reinforcement Learningcited by 51
  3. 2024LinGen: Towards High-Resolution Minute-Length Text-to-Video Generation with Linear Computational Complexitycited by 41
  4. 2024Autoregressive Models in Vision: A Surveycited by 39
  5. 2025VideoSSM: Autoregressive Long Video Generation with Hybrid State-Space Memorycited by 39
  6. 2025Lumina-mGPT 2.0: Stand-Alone AutoRegressive Image Modelingcited by 35
  7. 2025Bridging Continuous and Discrete Tokens for Autoregressive Visual Generationcited by 29
  8. 2024Is Your World Simulator a Good Story Presenter? A Consecutive Events-Based Benchmark for Future Long Video Generationcited by 28
  9. 2025Next Block Prediction: Video Generation via Semi-Autoregressive Modelingcited by 26
  10. 2024LVD-2M: A Long-take Video Dataset with Temporally Dense Captionscited by 24
  11. 2025A Survey of Interactive Generative Videocited by 23
  12. 2025Seeing What Matters: Generalizable AI-generated Video Detection with Forensic-Oriented Augmentationcited by 23
  13. 2025Code2Video: A Code-centric Paradigm for Educational Video Generationcited by 23
  14. 2025Lumos-1: On Autoregressive Video Generation with Discrete Diffusion from a Unified Model Perspectivecited by 22
  15. 2026ShotStream: Streaming Multi-Shot Video Generation for Interactive Storytellingcited by 22
  16. 2024InfinityDrive: Breaking Time Limits in Driving World Modelscited by 21
  17. 2025The Devil is in the Prompts: Retrieval-Augmented Prompt Optimization for Text-to-Video Generationcited by 21
  18. 2025NarrLV: Towards a Comprehensive Narrative-Centric Evaluation for Long Video Generationcited by 21
  19. 2025Error Analyses of Auto-Regressive Video Diffusion Models: A Unified Frameworkcited by 20
  20. 2025GenDoP: Auto-regressive Camera Trajectory Generation as a Director of Photographycited by 18
  21. 2026INSPATIO-WORLD: A Real-Time 4D World Simulator via Spatiotemporal Autoregressive Modelingcited by 18
  22. 2025VideoMAR: Autoregressive Video Generatio with Continuous Tokenscited by 15
  23. 2025Towards One-step Causal Video Generation via Adversarial Self-Distillationcited by 15
  24. 2026Streaming Autoregressive Video Generation via Diagonal Distillationcited by 15
  25. 2024Do Language Models Understand Time?cited by 14
  26. 2025Fine-Tuning Visual Autoregressive Models for Subject-Driven Generationcited by 13
  27. 2025BAgger: Backwards Aggregation for Mitigating Drift in Autoregressive Video Diffusion Modelscited by 13
  28. 2024Long Video Diffusion Generation with Segmented Cross-Attention and Content-Rich Video Data Curationcited by 12
  29. 2025UltraViCo: Breaking Extrapolation Limits in Video Diffusion Transformerscited by 12
  30. 2026StableWorld: Towards Stable and Consistent Long Interactive Video Generationcited by 12
  31. 2026MultiGen: Level-Design for Editable Multiplayer Worlds in Diffusion Game Enginescited by 12
  32. 2025Eliciting In-context Retrieval and Reasoning for Long-context Large Language Modelscited by 11
  33. 2025SpectralAR: Spectral Autoregressive Visual Generationcited by 11
  34. 2025Uniform Discrete Diffusion with Metric Path for Video Generationcited by 11
  35. 2024Towards Unifying Understanding and Generation in the Era of Vision Foundation Models: A Survey from the Autoregression Perspectivecited by 10
  36. 2025Conditional Panoramic Image Generation via Masked Autoregressive Modelingcited by 9
  37. 2025RAPO++: Cross-Stage Prompt Optimization for Text-to-Video Generation via Data Alignment and Test-Time Scalingcited by 9
  38. 2025BIFE: Better Interaction, Fewer Errors for Minute-Long Video Generationcited by 9

50 most cited of 99 All 99 (JSON)