mlsanityData to 14 Oct 2026

StreamingEffect: Real-Time Human-Centric Video Effect Generation

arXiv 2605.17019 ↗First posted 16 May 2026cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

Streaming video effect generation is highly desirable for live human-centric applications such as e-commerce streaming, entertainment, and vlogging, yet remains difficult due to the lack of suitable data and deployable editing models. Unlike generic video generation, this task requires real-time video-to-video editing that adds expressive effects while preserving human identity, background content, and temporal consistency. Existing acceleration efforts mainly focus on text-to-video generation, while efficient distillation for video editing remains largely underexplored. In this paper, we present StreamingEffect, a real-time human-centric streaming video effect framework. We adopt an in-context video editing architecture and train a high-quality bidirectional teacher, then distill it into a causal autoregressive student and further reduce sampling from 50 steps to 4 steps. We also introduce keyframe control, allowing reference effect frames to be injected online and propagated through the stream for interactive editing. To address the data bottleneck, we construct VideoEffect-130K, to our knowledge the largest human-centric video effect dataset, containing 70K effect videos and 60K editing videos across 600 effect categories curated from short-video and editing platforms. Experiments show that our method enables real-time, high-quality 720p video editing on a single H200 GPU.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 72 of its 75 references that are in the map. First posted on arXiv 16 May 2026.

Builds on

72 of 75 references.

References · 72

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2020Denoising Diffusion Implicit Modelscited by 9,781
  5. 2022Scalable Diffusion Models with Transformerscited by 6,054
  6. 2022Classifier-Free Diffusion Guidancecited by 5,847
  7. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  8. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  9. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  10. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  11. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  12. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
38 more
  1. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  2. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  3. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  4. 2023Consistency Modelscited by 1,822
  5. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  6. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  7. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  8. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  9. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
  10. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,178
  11. 2023One-step Diffusion with Distribution Matching Distillationcited by 1,019
  12. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 961
  13. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
  14. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 846
  15. 2023Adversarial Diffusion Distillationcited by 759
  16. 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 684
  17. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 681
  18. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 625
  19. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Modelscited by 579
  20. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 546
  21. 2025VACE: All-in-One Video Creation and Editingcited by 479
  22. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 466
  23. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 455
  24. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 442
  25. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 433
  26. 2023InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generationcited by 395
  27. 2023Pix2Video: Video Editing using Image Diffusioncited by 320
  28. 2023Video-P2P: Video Editing with Cross-attention Controlcited by 314
  29. 2023Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videoscited by 314
  30. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 255
  31. 2023Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translationcited by 253
  32. 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Timecited by 234
  33. 2024Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animationcited by 174
  34. 2024AnyV2V: A Tuning-Free Framework For Any Video-to-Video Editing Taskscited by 154
  35. 2025Diffusion Adversarial Post-Training for One-Step Video Generationcited by 151
  36. 2023SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generationcited by 143
  37. 2023CoDeF: Content Deformation Fields for Temporally Consistent Video Processingcited by 108
  38. 2023Consistent Video-to-Video Transfer Using Synthetic Datasetcited by 81

50 most cited of 72 All 72 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 8.

8 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026EditStream: A Unified Autoregressive Framework for Interactive Video Generation and Editing
  2. 2024Streaming Video Diffusion: Online Video Editing with Diffusion Models
  3. 2026VEFX-Bench: A Holistic Benchmark for Generic Video Editing and Visual Effects
  4. 2024VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generation
  5. 2026VideoArgus: Agentic Rubric-Grounded Unified Evaluation for Video Generation and Editing
  6. 2025MotionStream: Real-Time Video Generation with Interactive Motion Controls
  7. 2026Stream-T1: Test-Time Scaling for Streaming Video Generation
  8. 2025Reward Forcing: Efficient Streaming Video Generation with Rewarded Distribution Matching Distillation