mlsanityData to 14 Oct 2026

StreamDiT: Real-Time Streaming Text-to-Video Generation

arXiv 2507.03745 ↗First posted 4 Jul 2025cs.CV cs.AI cs.LGcited by 48

On the map: Diffusion models → Video generation

Abstract (arXiv)

Recently, great progress has been achieved in text-to-video (T2V) generation by scaling transformer-based diffusion models to billions of parameters, which can generate high-quality videos. However, existing models typically produce only short clips offline, restricting their use cases in interactive and real-time applications. This paper addresses these challenges by proposing StreamDiT, a streaming video generation model. StreamDiT training is based on flow matching by adding a moving buffer. We design mixed training with different partitioning schemes of buffered frames to boost both content consistency and visual quality. StreamDiT modeling is based on adaLN DiT with varying time embedding and window attention. To practice the proposed method, we train a StreamDiT model with 4B parameters. In addition, we propose a multistep distillation method tailored for StreamDiT. Sampling distillation is performed in each segment of a chosen partitioning scheme. After distillation, the total number of function evaluations (NFEs) is reduced to the number of chunks in a buffer. Finally, our distilled model reaches real-time performance at 16 FPS on one GPU, which can generate video streams at 512p resolution. We evaluate our method through both quantitative metrics and human evaluation. Our model enables real-time applications, e.g. streaming generation, interactive generation, and video-to-video. We provide video results and more examples in our project website: https://cumulo-autumn.github.io/StreamDiT/

Key facts. Cited by 48 papers in the map (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 42 of its 49 references that are in the map. First posted on arXiv 4 Jul 2025.

Builds on

42 of 49 references.

Closest ancestors · 19

Same direction, newest first

  1. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion9 Jun
  2. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Models10 Dec
  3. 2024Progressive Autoregressive Video Diffusion Models10 Oct
  4. 2024Video-Infinity: Distributed Long Video Generation24 Jun
  5. 2024ViD-GPT: Introducing GPT-style Autoregressive Generation in Video Diffusion Models16 Jun
  6. 2024VideoTetris: Towards Compositional Text-to-Video Generation6 Jun
  7. 2024FIFO-Diffusion: Generating Infinite Videos from Text without Training19 May
  8. 2024StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text21 Mar
  9. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models17 Jan
  10. 2023FreeNoise: Tuning-Free Longer Video Diffusion via Noise Rescheduling23 Oct
  11. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generation9 Oct
  12. 2023Reuse and Diffuse: Iterative Denoising for Text-to-Video Generation7 Sep
7 more
  1. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning10 Jul
  2. 2023Gen-L-Video: Multi-Text to Long Video Generation via Temporal Co-Denoising29 May
  3. 2023VDT: General-purpose Video Diffusion Transformers via Mask Modeling22 May
  4. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models18 Apr
  5. 2023NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generation22 Mar
  6. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generation23 Nov
  7. 2022Video Diffusion Models7 Apr

Other references · 23

  1. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  2. 2022Scalable Diffusion Models with Transformerscited by 6,054
  3. 2022Flow Matching for Generative Modelingcited by 5,702
  4. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  5. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
  6. 2023Consistency Modelscited by 1,822
  7. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  8. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  9. 2023One-step Diffusion with Distribution Matching Distillationcited by 1,019
  10. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 985
  11. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 846
  12. 2022On Distillation of Guided Diffusion Modelscited by 725
11 more
  1. 2024Autoregressive Image Generation without Vector Quantizationcited by 651
  2. 2024Movie Gen: A Cast of Media Foundation Modelscited by 644
  3. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 536
  4. 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 509
  5. 2022CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformerscited by 307
  6. 2022UL2: Unifying Language Learning Paradigmscited by 295
  7. 2023Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystackcited by 279
  8. 2024MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidancecited by 198
  9. 2023UFOGen: You Forward Once Large Scale Text-to-Image Generation via Diffusion GANscited by 166
  10. 2024Snap Video: Scaled Spatiotemporal Transformers for Text-to-Video Synthesiscited by 105
  11. 2023StreamDiffusion: A Pipeline-level Solution for Real-time Interactive Generationcited by 58

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 13; 2026: 35.

48 papers.

Sort:

By area:

  1. 2025LongLive: Real-time Interactive Long Video Generationcited by 265
  2. 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Timecited by 234
  3. 2025Self-Forcing++: Towards Minute-Scale High-Quality Video Generationcited by 192
  4. 2025Training Agents Inside of Scalable World Modelscited by 136
  5. 2025RELIC: Interactive Video World Model with Long-Horizon Memorycited by 96
  6. 2025Infinity-RoPE: Action-Controllable Infinite Video Generation Emerges From Autoregressive Self-Rolloutcited by 86
  7. 2025Stable Video Infinity: Infinite-Length Video Generation with Error Recyclingcited by 73
  8. 2026Context Forcing: Consistent Autoregressive Video Generation with Long Contextcited by 59
  9. 2026LoL: Longer than Longer, Scaling Video Generation to Hourcited by 32
  10. 2025StreamDiffusionV2: A Streaming System for Dynamic and Interactive Video Generationcited by 30
  11. 2026MemRoPE: Training-Free Infinite Video Generation via Evolving Memory Tokenscited by 27
  12. 2026Avatar Forcing: Real-Time Interactive Head Avatar Generation for Natural Conversationcited by 19
36 more
  1. 2025X-Streamer: Unified Human World Modeling with Audiovisual Interactioncited by 18
  2. 2026LongLive-RAG: A General Retrieval-Augmented Framework for Long Video Generationcited by 16
  3. 2026ABot-World-0: Infinite Interactive World Rollout on a Single Desktop GPUcited by 16
  4. 2026Light Forcing: Accelerating Autoregressive Video Diffusion via Sparse Attentioncited by 12
  5. 2026Mode Seeking meets Mean Seeking for Fast Long Video Generationcited by 12
  6. 2026Sparse Forcing: Native Trainable Sparse Attention for Real-time Autoregressive Diffusion Video Generationcited by 11
  7. 2026ReWorld: An Interactive World Model with Long-Horizon Memorycited by 10
  8. 2026Long-Horizon Streaming Video Generation via Hybrid Attention with Decoupled Distillationcited by 8
  9. 2026StreamingEffect: Real-Time Human-Centric Video Effect Generationcited by 8
  10. 2026Efficient Video Diffusion Models: Advancements and Challengescited by 7
  11. 2026FadeMem: Distance-Aware Memory Consolidation for Autoregressive Video Diffusioncited by 7
  12. 2026Reward-Forcing: Autoregressive Video Generation with Reward Feedbackcited by 6
  13. 2026AvatarForcing: One-Step Streaming Talking Avatars via Local-Future Sliding-Window Denoisingcited by 6
  14. 2026AlayaWorld: Long-Horizon and Playable Video World Generationcited by 6
  15. 2025Streaming Drag-Oriented Interactive Video Manipulation: Drag Anything, Anytime!cited by 5
  16. 2026Ring Forcing: Towards Precise Long-Term Memory for Autoregressive Video Diffusioncited by 5
  17. 2025AutoRefiner: Improving Autoregressive Video Diffusion Models via Reflective Refinement Over the Stochastic Sampling Pathcited by 4
  18. 2026Slot-ID: Identity-Preserving Video Generation from Reference Videos via Slot-Based Temporal Identity Encodingcited by 4
  19. 2026Surprise Forcing: What to Remember, When to Skip in Long Video Generationcited by 3
  20. 2025Live Avatar: Streaming Real-time Audio-Driven Avatar Generation with Infinite Lengthcited by 2
  21. 2026SARAH: Spatially Aware Real-time Agentic Humanscited by 2
  22. 2026Audio-Visual Intelligence in Large Foundation Modelscited by 2
  23. 2025SneakPeek: Future-Guided Instructional Streaming Video Generationcited by 1
  24. 2026Steady-Forcing: Balancing Spatial Persistence and Motion Continuity in Long-Horizon Nature Video Diffusioncited by 1
  25. 2026ReGenVC: End-to-End Real-Time Generative Video Coding at Ultra-Low Bitratecited by 1
  26. 2026ST-GDance++: A Scalable Spatial-Temporal Diffusion for Long-Duration Group Choreographycited by 0
  27. 2026OphEdit: Training-Free Text-Guided Editing of Ophthalmic Surgical Videoscited by 0
  28. 2026Inverting the Streaming-Diffusion Bottleneck: Video-Rate MLLM-Conditioned Edit Diffusion on a Consumer GPUcited by 0
  29. 2026UnStep: Training-Free Acceleration of Causal Video Diffusion with Fewer Steps Than Distillationcited by 0
  30. 2026Self-Aligned Forcing: Streaming Video Diffusion with Differentiable Noisy Historycited by 0
  31. 2026Online Evolution Strategy for Flow-Matching VLA Policies via Self-Supervised Trajectory Distribution Optimizationcited by 0
  32. 2026In-Distribution Forcing for Long Video Generation at Test Timecited by 0
  33. 2026VIS-Ground: Video Interactive Storytelling with Contextual Groundingcited by 0
  34. 2026REACT: Rolling Denoising and Dual Decoupling for Reactive Robot Control with VLA Modelscited by 0
  35. 2026LongForcing: Scaling Temporal Manifold Matching for Long-Horizon World Rolloutcited by 0
  36. 2026Speculative Sampling in Interactive Video Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Stream-T1: Test-Time Scaling for Streaming Video Generation
  2. 2025MotionStream: Real-Time Video Generation with Interactive Motion Controls
  3. 2024Grid Diffusion Models for Text-to-Video Generation
  4. 2023Dual-Stream Diffusion Net for Text-to-Video Generation
  5. 2026Stream Forcing: Constructing Unified Training Trajectory for Robust Streaming Video Generation
  6. 2024Looking Backward: Streaming Video-to-Video Translation with Feature Banks
  7. 2024Vidu: a Highly Consistent, Dynamic and Skilled Text-to-Video Generator with Diffusion Models
  8. 2024LinGen: Towards High-Resolution Minute-Length Text-to-Video Generation with Linear Computational Complexity