mlsanityData to 7 Oct 2026

StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text

arXiv 2403.14773 ↗First posted 21 Mar 2024cs.CV cs.AI cs.CLcited by 242

On the map: World models → Video-generation world models and simulators

also in 1 more
Abstract (arXiv)

Text-to-video diffusion models enable the generation of high-quality videos that follow text instructions, making it easy to create diverse and individual content. However, existing approaches mostly focus on high-quality short video generation (typically 16 or 24 frames), ending up with hard-cuts when naively extended to the case of long video synthesis. To overcome these limitations, we introduce StreamingT2V, an autoregressive approach for long video generation of 80, 240, 600, 1200 or more frames with smooth transitions. The key components are:(i) a short-term memory block called conditional attention module (CAM), which conditions the current generation on the features extracted from the previous chunk via an attentional mechanism, leading to consistent chunk transitions, (ii) a long-term memory block called appearance preservation module, which extracts high-level scene and object features from the first video chunk to prevent the model from forgetting the initial scene, and (iii) a randomized blending approach that enables to apply a video enhancer autoregressively for infinitely long videos without inconsistencies between chunks. Experiments show that StreamingT2V generates high motion amount. In contrast, all competing image-to-video methods are prone to video stagnation when applied naively in an autoregressive manner. Thus, we propose with StreamingT2V a high-quality seamless text-to-long video generator that outperforms competitors with consistency and motion. Our code will be available at: https://github.com/Picsart-AI-Research/StreamingT2V

Key facts. Cited by 242 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 45 of its 76 references that are in the map. First posted on arXiv 21 Mar 2024.

Builds on

45 of 76 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generation6 Feb
  2. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
  3. 2023Structure and Content-Guided Video Synthesis with Diffusion Models6 Feb
  4. 2021Learning Transferable Visual Models From Natural Language Supervision26 Feb
  5. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 40

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  8. 2022Classifier-Free Diffusion Guidancecited by 5,717
  9. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  10. 2017Neural Discrete Representation Learningcited by 4,650
  11. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  12. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
28 more
  1. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  2. 2022Video Diffusion Modelscited by 2,280
  3. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  4. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  5. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  6. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  7. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  8. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  9. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 666
  10. 2023ModelScope Text-to-Video Technical Reportcited by 651
  11. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488
  12. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  13. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 463
  14. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 461
  15. 2023I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Modelscited by 343
  16. 2023Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generationcited by 300
  17. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 254
  18. 2018Learning Blind Video Temporal Consistencycited by 247
  19. 2023SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Predictioncited by 231
  20. 2023FreeNoise: Tuning-Free Longer Video Diffusion via Noise Reschedulingcited by 173
  21. 2023SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Modelscited by 172
  22. 2023Make Pixels Dance: High-Dynamic Video Generationcited by 140
  23. 2023Gen-L-Video: Multi-Text to Long Video Generation via Temporal Co-Denoisingcited by 135
  24. 2023VideoGen: A Reference-Guided Latent Diffusion Approach for High Definition Text-to-Video Generationcited by 74
  25. 2023ARTboldsymbol·V: Auto-Regressive Text-to-Video Generation with Diffusion Modelscited by 74
  26. 2023AnimateAnything: Fine-Grained Open Domain Image Animation with Motion Guidancecited by 66
  27. 2023LivePhoto: Real Image Animation with Text-guided Motion Controlcited by 44
  28. 2023MEVG: Multi-event Video Generation with Text-to-Video Modelscited by 40

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 55; 2025: 89; 2026: 98.

242 papers.

Sort:

By area:

  1. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
  2. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 325
  3. 2025LongLive: Real-time Interactive Long Video Generationcited by 237
  4. 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Timecited by 209
  5. 2024VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generationcited by 190
  6. 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Surveycited by 176
  7. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 161
  8. 2025WorldMem: Long-term Consistent World Simulation with Memorycited by 152
  9. 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modelingcited by 152
  10. 2025Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Modelscited by 126
  11. 2025Video World Models with Long-term Spatial Memorycited by 124
  12. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
38 more
  1. 2025Long Context Tuning for Video Generationcited by 110
  2. 2024Adaptive Caching for Faster Video Generation with Diffusion Transformerscited by 100
  3. 2025One-Minute Video Generation with Test-Time Trainingcited by 100
  4. 2024Loong: Generating Minute-level Long Videos with Autoregressive Language Modelscited by 99
  5. 2024Hallo2: Long-Duration and High-Resolution Audio-Driven Portrait Image Animationcited by 95
  6. 2025Voyager: Long-Range and World-Consistent Video Diffusion for Explorable 3D Scene Generationcited by 93
  7. 2025Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Conditioncited by 92
  8. 2025Mixture of Contexts for Long Video Generationcited by 90
  9. 2025RELIC: Interactive Video World Model with Long-Horizon Memorycited by 89
  10. 2024FreeLong: Training-Free Long Video Generation with SpectralBlend Temporal Attentioncited by 87
  11. 2024DeMamba: AI-Generated Video Detection on Million-Scale GenVideo Benchmarkcited by 85
  12. 2024MovieDreamer: Hierarchical Generation for Coherent Long Visual Sequencecited by 85
  13. 2025Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generationcited by 84
  14. 2025Reward Forcing: Efficient Streaming Video Generation with Rewarded Distribution Matching Distillationcited by 75
  15. 2024DiTCtrl: Exploring Attention Control in Multi-Modal Diffusion Transformer for Tuning-Free Multi-Prompt Longer Video Generationcited by 73
  16. 2025SkyReels-A2: Compose Anything in Video Diffusion Transformerscited by 72
  17. 2024Ca2-VDM: Efficient Autoregressive Video Diffusion Model with Causal Generation and Cache Sharingcited by 70
  18. 2025Stable Video Infinity: Infinite-Length Video Generation with Error Recyclingcited by 68
  19. 2024Evaluation of Text-to-Video Generation Models: A Dynamics Perspectivecited by 66
  20. 2024Progressive Autoregressive Video Diffusion Modelscited by 65
  21. 2025Hunyuan-GameCraft-2: Instruction-following Interactive Game World Modelcited by 58
  22. 2024FreeTraj: Tuning-Free Trajectory Control in Video Diffusion Modelscited by 55
  23. 2024VideoTetris: Towards Compositional Text-to-Video Generationcited by 51
  24. 2025HoloCine: Holistic Generation of Cinematic Multi-Shot Long Video Narrativescited by 51
  25. 2025Memory Forcing: Spatio-Temporal Memory for Consistent Scene Generation on Minecraftcited by 46
  26. 2025End-to-End Training for Autoregressive Video Diffusion via Self-Resamplingcited by 45
  27. 2025StreamDiT: Real-Time Streaming Text-to-Video Generationcited by 43
  28. 2024ARLON: Boosting Diffusion Transformers with Autoregressive Models for Long Video Generationcited by 41
  29. 2025StableAvatar: Infinite-Length Audio-Driven Avatar Video Generationcited by 41
  30. 2025MemFlow: Flowing Adaptive Memory for Consistent and Efficient Long Video Narrativescited by 40
  31. 2024TrackGo: A Flexible and Efficient Method for Controllable Video Generationcited by 39
  32. 2024Mind the Time: Temporally-Controlled Multi-Event Video Generationcited by 39
  33. 2025VideoSSM: Autoregressive Long Video Generation with Hybrid State-Space Memorycited by 39
  34. 2025VRAG: Learning World Models for Interactive Video Generationcited by 37
  35. 2024LLMs Meet Multimodal Generation and Editing: A Surveycited by 35
  36. 2025OmniEraser: Remove Objects and Their Effects in Images with Paired Video-Frame Datacited by 35
  37. 2024DreamForge: Motion-Aware Autoregressive Video Generation for Multi-View Driving Scenescited by 34
  38. 2024ViD-GPT: Introducing GPT-style Autoregressive Generation in Video Diffusion Modelscited by 33

50 most cited of 242 All 242 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025MALT Diffusion: Memory-Augmented Latent Transformers for Any-Length Video Generation
  2. 2025Video Text Preservation with Synthetic Text-Rich Videos
  3. 2024xGen-VideoSyn-1: High-fidelity Text-to-Video Synthesis with Compressed Representations
  4. 2025TokensGen: Harnessing Condensed Tokens for Long Video Generation
  5. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models
  6. 2024Grid Diffusion Models for Text-to-Video Generation
  7. 2024Vivid-ZOO: Multi-View Video Generation with Diffusion Model
  8. 2026Towards Error-Free Long Video Generation