EvoStreaming: Your Offline Video Model Is a Natively Streaming Assistant
Not on the map. Why?
Abstract (arXiv)
Streaming video understanding demands more than watching longer videos: assistants must decide when to speak in real time, balancing responsiveness against verbosity. Yet most video-language models (VideoLLMs) are trained for offline inference, and existing streaming benchmarks externalize this timing decision to the evaluator. We address this gap with RealStreamEval, a frame-level multi-turn evaluation protocol that exposes models to sequential observations and penalizes unnecessary responses. Under this protocol, we observed that strong offline VideoLLMs retain useful visual understanding but lack an interaction policy for deciding when to respond. Motivated by this observation, we propose EvoStreaming, a self-evolved streaming adaptation framework in which the base model itself acts as data generator, relevance annotator, and roll-out policy to synthesize streaming trajectories without external supervision. With only 1,000 self-generated samples (139× less than the leading streaming instruction-tuning approach) and no architectural changes, EvoStreaming consistently improves the overall RealStreamEval score by up to 10.8 points across five open VideoLLM backbones (Qwen2/2.5/3-VL, InternVL-3.5, MiniCPM-V4.5) while largely preserving offline video performance. These results suggest that data-efficient interaction tuning is a practical path for adapting existing VideoLLMs to streaming assistants.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 62 of its 67 references that are in the map. First posted on arXiv 11 May 2026.
Builds on
62 of 67 references.
References · 62
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023Visual Instruction Tuningcited by 7,869
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024GPT-4o System Cardcited by 4,588
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
- 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
38 more
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
- 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
- 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
- 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 771
- 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 734
- 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
- 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 665
- 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
- 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 518
- 2024Long Context Transfer from Language to Visioncited by 515
- 2026Kimi K2.5: Visual Agentic Intelligencecited by 501
- 2024LVBench: An Extreme Long Video Understanding Benchmarkcited by 410
- 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understandingcited by 406
- 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 388
- 2025Kimi-VL Technical Reportcited by 388
- 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understandingcited by 356
- 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 355
- 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 353
- 2024MLVU: Benchmarking Multi-task Long Video Understandingcited by 292
- 2024VideoLLM-online: Online Video Large Language Model for Streaming Videocited by 225
- 2025MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipecited by 186
- 2024Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streamscited by 169
- 2025Stop Looking for Important Tokens in Multimodal Language Models: Duplication Matters Morecited by 155
- 2024StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understandingcited by 144
- 2025OVO-Bench: How Far is Your Video-LLMs from Real-World Online Video Understanding?cited by 142
- 2025Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reactioncited by 127
- 2025TimeChat-Online: 80% Visual Tokens are Naturally Redundant in Streaming Videoscited by 97
- 2025EfficientVLA: Training-Free Acceleration and Compression for Vision-Language-Action Modelscited by 93
- 2025Token Pruning in Multimodal Large Language Models: Are We Solving the Right Problem?cited by 76
- 2025StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistantcited by 63
- 2024Kangaroo: Lossless Self-Speculative Decoding via Double Early Exitingcited by 58
- 2025Video Compression Commander: Plug-and-Play Inference Acceleration for Video Large Language Modelscited by 38
- 2025Accelerating Streaming Video Large Language Models via Hierarchical Token Compressioncited by 37
- 2026LongCat-Next: Lexicalizing Modalities as Discrete Tokenscited by 36
- 2025Global Compression Commander: Plug-and-Play Inference Acceleration for High-Resolution Large Vision-Language Modelscited by 32
50 most cited of 62 All 62 (JSON)
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026OmniOPSD: Rationale-Privileged On-Policy Self-Distillation for Affective Computingcited by 6
- 2026LMM-Track4D: Eliciting 4D Dynamic Reasoning in LMMs via Trajectory-Grounded Dialoguecited by 2
- 2026How Well Can Your Video Model Remember? Measuring Memory-Budget Trade-offs in Long Video Understandingcited by 1
- 2026MoTE: Mixture of Task Experts for Multi-Task Video Understandingcited by 0
- 2026Learning to Reason with Persistent Object States for Video Instance Segmentationcited by 0
- 2026Learning to Reason with Persistent Object States for Video Instance Segmentation28 Sep 2026
- 2026MoTE: Mixture of Task Experts for Multi-Task Video Understanding25 Aug 2026
- 2026How Well Can Your Video Model Remember? Measuring Memory-Budget Trade-offs in Long Video Understanding17 Jun 2026
- 2026OmniOPSD: Rationale-Privileged On-Policy Self-Distillation for Affective Computing14 Jun 2026
- 2026LMM-Track4D: Eliciting 4D Dynamic Reasoning in LMMs via Trajectory-Grounded Dialogue19 May 2026