StreamingEffect: Real-Time Human-Centric Video Effect Generation
Not on the map. Why?
Abstract (arXiv)
Streaming video effect generation is highly desirable for live human-centric applications such as e-commerce streaming, entertainment, and vlogging, yet remains difficult due to the lack of suitable data and deployable editing models. Unlike generic video generation, this task requires real-time video-to-video editing that adds expressive effects while preserving human identity, background content, and temporal consistency. Existing acceleration efforts mainly focus on text-to-video generation, while efficient distillation for video editing remains largely underexplored. In this paper, we present StreamingEffect, a real-time human-centric streaming video effect framework. We adopt an in-context video editing architecture and train a high-quality bidirectional teacher, then distill it into a causal autoregressive student and further reduce sampling from 50 steps to 4 steps. We also introduce keyframe control, allowing reference effect frames to be injected online and propagated through the stream for interactive editing. To address the data bottleneck, we construct VideoEffect-130K, to our knowledge the largest human-centric video effect dataset, containing 70K effect videos and 60K editing videos across 600 effect categories curated from short-video and editing platforms. Experiments show that our method enables real-time, high-quality 720p video editing on a single H200 GPU.
Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 72 of its 75 references that are in the map. First posted on arXiv 16 May 2026.
Builds on
72 of 75 references.
References · 72
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2023GPT-4 Technical Reportcited by 18,396
- 2020Denoising Diffusion Implicit Modelscited by 9,781
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2022Classifier-Free Diffusion Guidancecited by 5,847
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
38 more
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
- 2023Consistency Modelscited by 1,822
- 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
- 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
- 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
- 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
- 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,178
- 2023One-step Diffusion with Distribution Matching Distillationcited by 1,019
- 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 961
- 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
- 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 846
- 2023Adversarial Diffusion Distillationcited by 759
- 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 684
- 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 681
- 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 625
- 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Modelscited by 579
- 2024Latte: Latent Diffusion Transformer for Video Generationcited by 546
- 2025VACE: All-in-One Video Creation and Editingcited by 479
- 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 466
- 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 455
- 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 442
- 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 433
- 2023InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generationcited by 395
- 2023Pix2Video: Video Editing using Image Diffusioncited by 320
- 2023Video-P2P: Video Editing with Cross-attention Controlcited by 314
- 2023Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videoscited by 314
- 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 255
- 2023Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translationcited by 253
- 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Timecited by 234
- 2024Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animationcited by 174
- 2024AnyV2V: A Tuning-Free Framework For Any Video-to-Video Editing Taskscited by 154
- 2025Diffusion Adversarial Post-Training for One-Step Video Generationcited by 151
- 2023SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generationcited by 143
- 2023CoDeF: Content Deformation Fields for Temporally Consistent Video Processingcited by 108
- 2023Consistent Video-to-Video Transfer Using Synthetic Datasetcited by 81
50 most cited of 72 All 72 (JSON)
Built on by
Citing papers per year: 2026: 8.
8 papers.
- 2026LiveEdit: Towards Real-Time Diffusion-Based Streaming Video Editingcited by 15
- 2026LiveLight: Real-time Streaming Video Relighting with Interactive Controlcited by 7
- 2026GeoEdit: Geometry-Aware Object Editing via Dual-Branch Denoisingcited by 1
- 2026MSEditor: Toward Consistent Multi-Shot Video Editingcited by 1
- 2026SubjectAnchor: Subject-Aware Memory-to-Video for Multi-Shot Storytellingcited by 1
- 2026MagicPrompt: Ultra-Lightweight Prompt Tuning for Video Generationcited by 0
- 2026Edit-VAR: Taming Visual Autoregressive Model for Precise Video Editingcited by 0
- 2026LongLive-Plug: Once-for-All Distillation for Video Generationcited by 0
- 2026LongLive-Plug: Once-for-All Distillation for Video Generation29 Sep 2026
- 2026SubjectAnchor: Subject-Aware Memory-to-Video for Multi-Shot Storytelling28 Sep 2026
- 2026Edit-VAR: Taming Visual Autoregressive Model for Precise Video Editing18 Sep 2026
- 2026MSEditor: Toward Consistent Multi-Shot Video Editing18 Aug 2026
- 2026LiveLight: Real-time Streaming Video Relighting with Interactive Control3 Aug 2026
- 2026MagicPrompt: Ultra-Lightweight Prompt Tuning for Video Generation16 Jul 2026
- 2026GeoEdit: Geometry-Aware Object Editing via Dual-Branch Denoising29 Jun 2026
- 2026LiveEdit: Towards Real-Time Diffusion-Based Streaming Video Editing25 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2026EditStream: A Unified Autoregressive Framework for Interactive Video Generation and Editing
- 2024Streaming Video Diffusion: Online Video Editing with Diffusion Models
- 2026VEFX-Bench: A Holistic Benchmark for Generic Video Editing and Visual Effects
- 2024VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generation
- 2026VideoArgus: Agentic Rubric-Grounded Unified Evaluation for Video Generation and Editing
- 2025MotionStream: Real-Time Video Generation with Interactive Motion Controls
- 2026Stream-T1: Test-Time Scaling for Streaming Video Generation
- 2025Reward Forcing: Efficient Streaming Video Generation with Rewarded Distribution Matching Distillation