mlsanityData to 14 Oct 2026

StreamDiffusionV2: A Streaming System for Dynamic and Interactive Video Generation

arXiv 2511.07399 ↗First posted 10 Nov 2025cs.CV cs.LGcited by 30

On the map: Diffusion models → Video generation

Abstract (arXiv)

Generative models are reshaping the live-streaming industry by redefining how content is created, styled, and delivered. Previous image-based streaming diffusion models have powered efficient and creative live streaming products but have hit limits on temporal consistency due to the foundation of image-based designs. Recent advances in video diffusion have markedly improved temporal consistency and sampling efficiency for offline generation. However, offline generation systems primarily optimize throughput by batching large workloads. In contrast, live online streaming operates under strict service-level objectives (SLOs): time-to-first-frame must be minimal, and every frame must meet a per-frame deadline with low jitter. Besides, scalable multi-GPU serving for real-time streams remains largely unresolved so far. To address this, we present StreamDiffusionV2, a training-free pipeline for interactive live streaming with video diffusion models. StreamDiffusionV2 integrates an SLO-aware batching scheduler and a block scheduler, together with a sink-token--guided rolling KV cache, a motion-aware noise controller, and other system-level optimizations. Moreover, we introduce a scalable pipeline orchestration that parallelizes the diffusion process across denoising steps and network layers, achieving near-linear FPS scaling without violating latency guarantees. The system scales seamlessly across heterogeneous GPU environments and supports flexible denoising steps (e.g., 1--4), enabling both ultra-low-latency and higher-quality modes. Without TensorRT or quantization, StreamDiffusionV2 renders the first frame within 0.5s and attains 58.28 FPS with a 14B-parameter model and 64.52 FPS with a 1.3B-parameter model on four H100 GPUs, making state-of-the-art generative live streaming practical and accessible--from individual creators to enterprise-scale platforms.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 45 of its 49 references that are in the map. First posted on arXiv 10 Nov 2025.

Builds on

45 of 49 references.

Closest ancestors · 13

Same direction, newest first

  1. 2025SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformer29 Sep
  2. 2025Matrix-game 2.0: An open-source real-time and streaming interactive world model18 Aug
  3. 2025StreamDiT: Real-Time Streaming Text-to-Video Generation4 Jul
  4. 2025Less is Enough: Training-Free Video Diffusion Acceleration via Runtime-Adaptive Caching3 Jul
  5. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion9 Jun
  6. 2025Long-Context State-Space Video World Models26 May
  7. 2025Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutation24 May
  8. 2025SkyReels-V2: Infinite-length Film Generative Model17 Apr
  9. 2025Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity3 Feb
  10. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Models10 Dec
  11. 2024Timestep Embedding Tells: It's Time to Cache for Video Diffusion Model28 Nov
  12. 2024Diffusion Models Are Real-Time Game Engines27 Aug
1 more
  1. 2024Looking Backward: Streaming Video-to-Video Translation with Feature Banks24 May

Other references · 32

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  3. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  4. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  5. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  6. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
  7. 2023One-step Diffusion with Distribution Matching Distillationcited by 1,019
  8. 2022On Distillation of Guided Diffusion Modelscited by 725
  9. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 703
  10. 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 684
  11. 2023Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusioncited by 459
  12. 2023Ring Attention with Blockwise Transformers for Near-Infinite Contextcited by 457
20 more
  1. 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 393
  2. 2024SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformerscited by 340
  3. 2024Simplifying, Stabilizing and Scaling Continuous-Time Consistency Modelscited by 283
  4. 2025MAGI-1: Autoregressive Video Generation at Scalecited by 276
  5. 2025LongLive: Real-time Interactive Long Video Generationcited by 265
  6. 2023DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Modelscited by 245
  7. 2024Deep Compression Autoencoder for Efficient High-Resolution Diffusion Modelscited by 217
  8. 2024Delta-DiT: A Training-Free Acceleration Method Tailored for Diffusion Transformerscited by 175
  9. 2024SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Accelerationcited by 160
  10. 2023Cache Me if You Can: Accelerating Diffusion Models through Block Cachingcited by 151
  11. 2024FORA: Fast-Forward Caching in Diffusion Transformer Accelerationcited by 137
  12. 2024SageAttention2: Efficient Attention with Thorough Outlier Smoothing and Per-thread INT4 Quantizationcited by 114
  13. 2023Parallel Sampling of Diffusion Modelscited by 112
  14. 2025One-Minute Video Generation with Test-Time Trainingcited by 101
  15. 2024DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Modelscited by 98
  16. 2024Vript: A Video Is Worth Thousands of Wordscited by 71
  17. 2023StreamDiffusion: A Pipeline-level Solution for Real-time Interactive Generationcited by 58
  18. 2024xDiT: an Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelismcited by 56
  19. 2024PipeFusion: Patch-level Pipeline Parallelism for Diffusion Transformers Inferencecited by 28
  20. 2024Balancing Pipeline Parallelism with Vocabulary Parallelismcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 26.

30 papers.

Sort:

By area:

  1. 2026Helios: Real Real-Time Long Video Generation Modelcited by 65
  2. 2026Infinite Worlds with Versatile Interactionscited by 48
  3. 2026Quant VideoGen: Auto-Regressive Long Video Generation via 2-Bit KV-Cache Quantizationcited by 23
  4. 2025TeleWorld: Towards Dynamic Multimodal Synthesis with a 4D World Modelcited by 16
  5. 2026LiveEdit: Towards Real-Time Diffusion-Based Streaming Video Editingcited by 15
  6. 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyondcited by 14
  7. 2026Alaya-EVOKE: From Linear-Scaling Supervision to Endless Worldcited by 14
  8. 2026SVG-EAR: Parameter-Free Linear Compensation for Sparse Video Generation via Error-aware Routingcited by 10
  9. 2025BIFE: Better Interaction, Fewer Errors for Minute-Long Video Generationcited by 9
  10. 2025EgoEdit: Dataset, Real-Time Streaming Model, and Benchmark for Egocentric Video Editingcited by 8
  11. 2026JoyAI-Video-Edit: Real-Time Open-Ended Video Editing with Autoregressive Diffusioncited by 6
  12. 2026Vidu S2: Real-Time Interactive, Editable, and Spatial Video Generationcited by 5
18 more
  1. 2026Adapting VACE for Real-Time Autoregressive Video Diffusioncited by 4
  2. 2026Vidu S1: A Real-Time Interactive Video Generation Modelcited by 4
  3. 2026Instruction-Based Video Editing by Repurposing an Image Editing Modelcited by 3
  4. 2025StreamFlow: Theory, Algorithm, and Implementation for High-Efficiency Rectified Flow Generationcited by 1
  5. 2026Video Compression Meets Video Generation: Latent Inter-Frame Pruning with Attention Recoverycited by 1
  6. 2026Video DeltaNet: A Video-Native Hybrid Attention for Livestream Video Generationcited by 1
  7. 2026StreamWise: Serving Multi-Modal Generation in Real-Time at Scalecited by 0
  8. 2026Q-ARVD: Quantizing Autoregressive Video Diffusion Modelscited by 0
  9. 2026Inverting the Streaming-Diffusion Bottleneck: Video-Rate MLLM-Conditioned Edit Diffusion on a Consumer GPUcited by 0
  10. 2026Stateful Worlds, Stateless Elasticity: Exact-State Serving for Interactive World Modelscited by 0
  11. 2026InfinityEdit: Infinite Video Editing with a Lightweight Edit-Ignition Adaptercited by 0
  12. 2026EditaLive! Unified Character Video Editing for Live Streamingcited by 0
  13. 2026Rethinking Streaming Video Diffusion Model: Context, Execution, and Trainingcited by 0
  14. 2026Streaming Video Editing with Easy Adaptationcited by 0
  15. 2026WorldAttention: An Efficient Attention Architecture for Interactive Video World Modelscited by 0
  16. 2026Weave Forcing: Compositional Memory Routing for Interactive Long Video Generationcited by 0
  17. 2026A Spatiotemporal Semantic Importance-Guided Unified Compression and Editing Framework for AI-Generated Videoscited by 0
  18. 2026VIS-Ground: Video Interactive Storytelling with Contextual Groundingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Stream-T1: Test-Time Scaling for Streaming Video Generation
  2. 2026Speculative Sampling in Interactive Video Generation
  3. 2026TurboServe: Serving Streaming Video Generation Efficiently and Economically
  4. 2025MotionStream: Real-Time Video Generation with Interactive Motion Controls
  5. 2026Stream Forcing: Constructing Unified Training Trajectory for Robust Streaming Video Generation
  6. 2024Streaming Video Diffusion: Online Video Editing with Diffusion Models
  7. 2026Visko Orbis 1.0: A Live Model for Real-Time Interactive Long Video Generation
  8. 2025Live Avatar: Streaming Real-time Audio-Driven Avatar Generation with Infinite Length