mlsanityData to 7 Oct 2026

NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generation

arXiv 2303.12346 ↗First posted 22 Mar 2023cs.CV cs.AIcited by 170

On the map: Diffusion models → Video generation

Abstract (arXiv)

In this paper, we propose NUWA-XL, a novel Diffusion over Diffusion architecture for eXtremely Long video generation. Most current work generates long videos segment by segment sequentially, which normally leads to the gap between training on short videos and inferring long videos, and the sequential generation is inefficient. Instead, our approach adopts a ``coarse-to-fine'' process, in which the video can be generated in parallel at the same granularity. A global diffusion model is applied to generate the keyframes across the entire time range, and then local diffusion models recursively fill in the content between nearby frames. This simple yet effective strategy allows us to directly train on long videos (3376 frames) to reduce the training-inference gap, and makes it possible to generate all segments in parallel. To evaluate our model, we build FlintstonesHD dataset, a new benchmark for long video generation. Experiments show that our model not only generates high-quality long videos with both global and local coherence, but also decreases the average inference time from 7.55min to 26s (by 94.26%) at the same hardware setting when generating 1024 frames. The homepage link is urlhttps://msra-nuwa.azurewebsites.net/

Key facts. Cited by 170 papers in the map (data through 7 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 30 of its 35 references that are in the map. First posted on arXiv 22 Mar 2023.

Builds on

30 of 35 references.

Closest ancestors · 5

Same direction, newest first

  1. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Models20 Nov
  2. 2022Flexible Diffusion Modeling of Long Videos23 May
  3. 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolation19 May
  4. 2022Video Diffusion Models7 Apr
  5. 2022Diffusion Probabilistic Modeling for Video Generation16 Mar

Other references · 25

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  5. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  6. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  7. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  8. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  9. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  10. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  11. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  12. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
13 more
  1. 2022GIT: A Generative Image-to-text Transformer for Vision and Languagecited by 536
  2. 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 523
  3. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  4. 2016Generating Videos with Scene Dynamicscited by 376
  5. 2022CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformerscited by 305
  6. 2021GODIVA: Generating Open-DomaIn Videos from nAtural Descriptionscited by 254
  7. 2021N"UWA: Visual Synthesis Pre-training for Neural visUal World creAtioncited by 253
  8. 2022Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformercited by 244
  9. 2016Temporal Generative Adversarial Nets with Singular Value Clippingcited by 198
  10. 2017Video Generation From Textcited by 153
  11. 2018To Create What You Tell: Generating Videos from Captionscited by 84
  12. 2022NUWA-Infinity: Autoregressive over Autoregressive Generation for Infinite Visual Synthesiscited by 71
  13. 2016Sync-DRAW: Automatic Video Generation using Deep Recurrent Attentive Architecturescited by 44

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 23; 2024: 66; 2025: 60; 2026: 21.

170 papers.

Sort:

By area:

  1. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  2. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
  3. 2023Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generationcited by 300
  4. 2023DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectorycited by 260
  5. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 254
  6. 2025LongLive: Real-time Interactive Long Video Generationcited by 237
  7. 2023SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Predictioncited by 231
  8. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  9. 2023A Survey on Video Diffusion Modelscited by 203
  10. 2024MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidancecited by 196
  11. 2025Matrix-game 2.0: An open-source real-time and streaming interactive world modelcited by 174
  12. 2023FreeNoise: Tuning-Free Longer Video Diffusion via Noise Reschedulingcited by 173
38 more
  1. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 161
  2. 2024MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captionscited by 144
  3. 2023Make Pixels Dance: High-Dynamic Video Generationcited by 140
  4. 2023VideoDirectorGPT: Consistent Multi-scene Video Generation via LLM-Guided Planningcited by 138
  5. 2023Gen-L-Video: Multi-Text to Long Video Generation via Temporal Co-Denoisingcited by 135
  6. 2024FIFO-Diffusion: Generating Infinite Videos from Text without Trainingcited by 134
  7. 2025Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Modelscited by 126
  8. 2025Epona: Autoregressive Diffusion World Model for Autonomous Drivingcited by 115
  9. 2025Long Context Tuning for Video Generationcited by 110
  10. 2024Snap Video: Scaled Spatiotemporal Transformers for Text-to-Video Synthesiscited by 105
  11. 2024Adaptive Caching for Faster Video Generation with Diffusion Transformerscited by 100
  12. 2025One-Minute Video Generation with Test-Time Trainingcited by 100
  13. 2024Loong: Generating Minute-level Long Videos with Autoregressive Language Modelscited by 99
  14. 2025Voyager: Long-Range and World-Consistent Video Diffusion for Explorable 3D Scene Generationcited by 93
  15. 2025Mixture of Contexts for Long Video Generationcited by 90
  16. 2024FreeLong: Training-Free Long Video Generation with SpectralBlend Temporal Attentioncited by 87
  17. 2024MovieDreamer: Hierarchical Generation for Coherent Long Visual Sequencecited by 85
  18. 2025Automated Movie Generation via Multi-Agent CoT Planningcited by 82
  19. 2023Intelligent Grimm -- Open-ended Visual Storytelling via Latent Diffusion Modelscited by 76
  20. 2025Reward Forcing: Efficient Streaming Video Generation with Rewarded Distribution Matching Distillationcited by 75
  21. 2025SkyReels-A2: Compose Anything in Video Diffusion Transformerscited by 72
  22. 2024Vlogger: Make Your Dream A Vlogcited by 70
  23. 2024Sonic: Shifting Focus to Global Audio Perception in Portrait Animationcited by 70
  24. 2024Progressive Autoregressive Video Diffusion Modelscited by 65
  25. 2024Ctrl-Adapter: An Efficient and Versatile Framework for Adapting Diverse Controls to Any Diffusion Modelcited by 64
  26. 2024VideoStudio: Generating Consistent-Content and Multi-Scene Videoscited by 62
  27. 2023GPT4Motion: Scripting Physical Motions in Text-to-Video Generation via Blender-Oriented GPT Planningcited by 61
  28. 2024FreeTraj: Tuning-Free Trajectory Control in Video Diffusion Modelscited by 55
  29. 2024Sora as a World Model? A Complete Survey on Text-to-Video Generationcited by 53
  30. 2023AutoDecoding Latent 3D Diffusion Modelscited by 50
  31. 2024Towards A Better Metric for Text-to-Video Generationcited by 50
  32. 2024EG4D: Explicit Generation of 4D Object without Score Distillationcited by 45
  33. 2023LivePhoto: Real Image Animation with Text-guided Motion Controlcited by 44
  34. 2025StreamDiT: Real-Time Streaming Text-to-Video Generationcited by 43
  35. 2024ARLON: Boosting Diffusion Transformers with Autoregressive Models for Long Video Generationcited by 41
  36. 2025StableAvatar: Infinite-Length Audio-Driven Avatar Video Generationcited by 41
  37. 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspectivecited by 41
  38. 2024A Survey on Long Video Generation: Challenges, Methods, and Prospectscited by 40

50 most cited of 170 All 170 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Efficient Video Diffusion Models: Advancements and Challenges
  2. 2025AR-Diffusion: Asynchronous Video Generation with Auto-Regressive Diffusion
  3. 2024Fast and Memory-Efficient Video Diffusion Using Streamlined Inference
  4. 2026Diff-VF: Training-free High-quality Long Video Generation via Diffusion Model
  5. 2025Playing with Transformer at 30+ FPS via Next-Frame Diffusion
  6. 2025Generating, Fast and Slow: Scalable Parallel Video Generation with Video Interface Networks
  7. 2026FlowLong: Inference-time Long Video Generation via Manifold-constrained Tweedie Matching
  8. 2025Survey of Video Diffusion Models: Foundations, Implementations, and Applications