mlsanityData to 7 Oct 2026

Progressive Autoregressive Video Diffusion Models

arXiv 2410.08151 ↗First posted 10 Oct 2024cs.CV cs.LGcited by 65

On the map: Diffusion models → Video generation

Abstract (arXiv)

Current frontier video diffusion models have demonstrated remarkable results at generating high-quality videos. However, they can only generate short video clips, normally around 10 seconds or 240 frames, due to computation limitations during training. Existing methods naively achieve autoregressive long video generation by directly placing the ending of the previous clip at the front of the attention window as conditioning, which leads to abrupt scene changes, unnatural motion, and error accumulation. In this work, we introduce a more natural formulation of autoregressive long video generation by revisiting the noise level assumption in video diffusion models. Our key idea is to 1. assign the frames with per-frame, progressively increasing noise levels rather than a single noise level and 2. denoise and shift the frames in small intervals rather than all at once. This allows for smoother attention correspondence among frames with adjacent noise levels, larger overlaps between the attention windows, and better propagation of information from the earlier to the later frames. Video diffusion models equipped with our progressive noise schedule can autoregressively generate long videos with much improved fidelity compared to the baselines and minimal quality degradation over time. We present the first results on text-conditioned 60-second (1440 frames) long video generation at a quality close to frontier models. Code and video results are available at https://desaixie.github.io/pa-vdm/.

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 40 of its 61 references that are in the map. First posted on arXiv 10 Oct 2024.

Builds on

40 of 61 references.

Closest ancestors · 20

Same direction, newest first

  1. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
  2. 2024FreeLong: Training-Free Long Video Generation with SpectralBlend Temporal Attention29 Jul
  3. 2024ViD-GPT: Introducing GPT-style Autoregressive Generation in Video Diffusion Models16 Jun
  4. 2024VideoTetris: Towards Compositional Text-to-Video Generation6 Jun
  5. 2024FIFO-Diffusion: Generating Infinite Videos from Text without Training19 May
  6. 2024StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generation2 May
  7. 2024Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance21 Mar
  8. 2024StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text21 Mar
  9. 2024Latte: Latent Diffusion Transformer for Video Generation5 Jan
  10. 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation28 Nov
  11. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
  12. 2023SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Prediction31 Oct
8 more
  1. 2023FreeNoise: Tuning-Free Longer Video Diffusion via Noise Rescheduling23 Oct
  2. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generation9 Oct
  3. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning10 Jul
  4. 2023Gen-L-Video: Multi-Text to Long Video Generation via Temporal Co-Denoising29 May
  5. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models18 Apr
  6. 2023NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generation22 Mar
  7. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation22 Dec
  8. 2022Imagen Video: High Definition Video Generation with Diffusion Models5 Oct

Other references · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  7. 2022Scalable Diffusion Models with Transformerscited by 5,931
  8. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  9. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  10. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  11. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  12. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
8 more
  1. 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 516
  2. 2023InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generationcited by 387
  3. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 254
  4. 2024EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditionscited by 233
  5. 2024MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captionscited by 144
  6. 2024Rolling Diffusion Modelscited by 102
  7. 2024Vlogger: Make Your Dream A Vlogcited by 70
  8. 2023Carve3D: Improving Multi-view Reconstruction Consistency for Diffusion Models with RL Finetuningcited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 34; 2026: 26.

65 papers.

Sort:

By area:

  1. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
  2. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
  3. 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Timecited by 209
  4. 2025RELIC: Interactive Video World Model with Long-Horizon Memorycited by 89
  5. 2025Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generationcited by 84
  6. 2024GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Controlcited by 76
  7. 2025MotionStream: Real-Time Video Generation with Interactive Motion Controlscited by 71
  8. 2025Memory Forcing: Spatio-Temporal Memory for Consistent Scene Generation on Minecraftcited by 46
  9. 2025End-to-End Training for Autoregressive Video Diffusion via Self-Resamplingcited by 45
  10. 2025StreamDiT: Real-Time Streaming Text-to-Video Generationcited by 43
  11. 2025ATI: Any Trajectory Instruction for Controllable Video Generationcited by 42
  12. 2024LinGen: Towards High-Resolution Minute-Length Text-to-Video Generation with Linear Computational Complexitycited by 41
38 more
  1. 2024Autoregressive Models in Vision: A Surveycited by 39
  2. 2025VRAG: Learning World Models for Interactive Video Generationcited by 37
  3. 2025World Models for Learning Dexterous Hand-Object Interactions from Human Videoscited by 26
  4. 2025Code2Video: A Code-centric Paradigm for Educational Video Generationcited by 23
  5. 2025MiLA: Multi-view Intensive-fidelity Long-term Video Generation World Model for Autonomous Drivingcited by 18
  6. 2026INSPATIO-WORLD: A Real-Time 4D World Simulator via Spatiotemporal Autoregressive Modelingcited by 18
  7. 2025Generative Pre-trained Autoregressive Diffusion Transformercited by 16
  8. 2025MALT Diffusion: Memory-Augmented Latent Transformers for Any-Length Video Generationcited by 14
  9. 2025Survey of Video Diffusion Models: Foundations, Implementations, and Applicationscited by 14
  10. 2025FloodDiffusion: Tailored Diffusion Forcing for Streaming Motion Generationcited by 14
  11. 2026LIVE: Long-horizon Interactive Video World Modelingcited by 14
  12. 2026ABot-World-0: Infinite Interactive World Rollout on a Single Desktop GPUcited by 13
  13. 2025MaskFlow: Discrete Flows For Flexible and Efficient Long Video Generationcited by 12
  14. 2026WorldBench: Benchmarking Physical Understanding of World Models by Isolating Physics Conceptscited by 12
  15. 2026WorldCam: Interactive Autoregressive 3D Gaming Worlds with Camera Pose as a Unifying Geometric Representationcited by 12
  16. 2025Physical Autoregressive Model for Robotic Manipulation without Action Pretrainingcited by 11
  17. 2024Towards Unifying Understanding and Generation in the Era of Vision Foundation Models: A Survey from the Autoregression Perspectivecited by 10
  18. 2025FastCar: Cache Attentive Replay for Fast Auto-Regressive Video Generation on the Edgecited by 9
  19. 2025From Virtual Games to Real-World Playcited by 9
  20. 2025Stream-DiffVSR: Low-Latency Streamable Video Super-Resolution via Auto-Regressive Diffusioncited by 7
  21. 2026Latent Spatial Memory for Video World Modelscited by 7
  22. 2025Tuning-Free Multi-Event Long Video Generation via Synchronized Coupled Samplingcited by 6
  23. 2025PersonaLive! Expressive Portrait Image Animation for Live Streamingcited by 6
  24. 2025WorldWarp: Propagating 3D Geometry with Asynchronous Video Diffusioncited by 6
  25. 2026Evolution of Video Generative Foundationscited by 6
  26. 2026Advancing Narrative Long Video Generation via Training-Free Identity-Aware Memorycited by 6
  27. 2026Video Generation Models as World Models: Efficient Paradigms, Architectures and Algorithmscited by 5
  28. 2025AutoRefiner: Improving Autoregressive Video Diffusion Models via Reflective Refinement Over the Stochastic Sampling Pathcited by 4
  29. 2026Towards Holistic Modeling for Video Frame Interpolation with Auto-regressive Diffusion Transformerscited by 4
  30. 2026Vidu S1: A Real-Time Interactive Video Generation Modelcited by 4
  31. 2026DistillAlign: Coordinating Mode Covering and Mode Seeking in Autoregressive Video Distillationcited by 4
  32. 2025Flow marching for a generative PDE foundation modelcited by 3
  33. 2026In-Context Forcing: Uncovering Context Effects in Autoregressive Video Diffusioncited by 3
  34. 2025LVTINO: LAtent Video consisTency INverse sOlver for High Definition Video Restorationcited by 2
  35. 2025UniTS: Unified Spatio-Temporal Generative Model for Remote Sensingcited by 2
  36. 2025Flowception: Temporally Expansive Flow Matching for Video Generationcited by 2
  37. 2026Latent Generative Solvers for Generalizable Long-Term Physics Simulationcited by 2
  38. 2026SoulX-LiveAct: Towards Hour-Scale Real-Time Human Animation with Neighbor Forcing and ConvKV Memorycited by 2

50 most cited of 65 All 65 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2022Diffusion Probabilistic Modeling for Video Generation
  2. 2025AR-Diffusion: Asynchronous Video Generation with Auto-Regressive Diffusion
  3. 2022Video Diffusion Models
  4. 2024Efficient Video Diffusion Models via Content-Frame Motion-Latent Decomposition
  5. 2025Playing with Transformer at 30+ FPS via Next-Frame Diffusion
  6. 2024Video Diffusion Models: A Survey
  7. 2026Efficient Video Diffusion Models: Advancements and Challenges
  8. 2022Flexible Diffusion Modeling of Long Videos