StreamPro: From Reactive Perception to Proactive Decision-Making in Streaming Video
Not on the map. Why?
Abstract (arXiv)
Proactive streaming video understanding requires models to continuously process video streams and decide when to respond, rather than merely what to respond. This naturally introduces a decision-making problem under partial observations, where models must balance early prediction against sufficient evidence. However, existing benchmarks largely follow a "see-then-answer" paradigm, where responses are triggered only after explicit evidence appears, effectively reducing proactive reasoning to delayed perception. As a result, they fail to evaluate a model's ability to make timely and reliable decisions under incomplete observations. Moreover, training proactive models is inherently challenging due to the extreme imbalance between silence and response signals in streaming trajectories, as well as the need to jointly optimize response correctness and timing. To address these challenges, we introduce StreamPro-Bench, a new benchmark that evaluates streaming models from three complementary perspectives: Perception Understanding, Temporal Reasoning, and Proactive Agency, where the last measures a model's ability to make early yet reliable decisions under partial observations. We further propose StreamPro, a two-stage training framework for proactive learning. First, we introduce CB-Stream Loss to mitigate the severe supervision imbalance during supervised fine-tuning (SFT). Then, we apply Group Relative Policy Optimization (GRPO) with a multi-grained reward design that involves both turn-level and trajectory-level rewards. Experiments show that StreamPro significantly improves proactive performance. On StreamPro-Bench, it achieves 41.5, substantially outperforming the previous best (10.4), while also maintaining strong performance on real-time streaming benchmarks, achieving 78.9 on StreamingBench-RTVU.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 39 of its 45 references that are in the map. First posted on arXiv 11 May 2026.
Builds on
39 of 45 references.
References · 39
- 2023GPT-4 Technical Reportcited by 18,132
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
- 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,239
27 more
- 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
- 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 644
- 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 550
- 2024VideoLLM-online: Online Video Large Language Model for Streaming Videocited by 216
- 2024VideoChat-Flash: Hierarchical Compression for Long-Context Video Modelingcited by 212
- 2024StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understandingcited by 135
- 2025OVO-Bench: How Far is Your Video-LLMs from Real-World Online Video Understanding?cited by 134
- 2025Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reactioncited by 122
- 2025TimeChat-Online: 80% Visual Tokens are Naturally Redundant in Streaming Videoscited by 90
- 2025StreamingVLM: Real-Time Understanding for Infinite Video Streamscited by 89
- 2025Flash-VStream: Efficient Real-Time Understanding for Long Video Streamscited by 78
- 2025StreamForest: Efficient Online Video Understanding with Persistent Event Memorycited by 71
- 2024E.T. Bench: Towards Open-Ended Event-Level Video-Language Understandingcited by 68
- 2025StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistantcited by 56
- 2024Advancing Video Anomaly Detection: A Concise Review and a New Datasetcited by 48
- 2025OmniMMI: A Comprehensive Multi-modal Interaction Benchmark in Streaming Video Contextscited by 48
- 2024VideoLLM Knows When to Speak: Enhancing Time-Sensitive Video Comprehension with Video-Text Duet Interaction Formatcited by 44
- 2025StreamMind: Unlocking Full Frame Rate Streaming Video Dialogue through Event-Gated Cognitioncited by 44
- 2025ViSpeak: Visual Instruction Feedback in Streaming Videoscited by 38
- 2025StreamAgent: Towards Anticipatory Agents for Streaming Video Understandingcited by 29
- 2025Eyes Wide Open: Ego Proactive Video-LLM for Streaming Videocited by 25
- 2025Streaming Video Instruction Tuningcited by 25
- 2025ProactiveVideoQA: A Comprehensive Benchmark Evaluating Proactive Interactions in Video Large Language Modelscited by 24
- 2025MMDuet2: Enhancing Proactive Interaction of Video MLLMs with Multi-Turn Reinforcement Learningcited by 18
- 2026AURA: Always-On Understanding and Real-Time Assistance via Video Streamscited by 15
- 2026Thinking in Streaming Videocited by 14
- 2026Progressive Online Video Understanding with Evidence-Aligned Timing and Transparent Decisionscited by 5
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026How Well Can Your Video Model Remember? Measuring Memory-Budget Trade-offs in Long Video Understandingcited by 1
- 2026LiveProBench: Can Streaming Video Models Really Interact Like Humans?cited by 0
- 2026Live Assistant: Learning Whether, When, and Whom to Assist in Real-World Live Social Streamscited by 0
- 2026Live Assistant: Learning Whether, When, and Whom to Assist in Real-World Live Social Streams23 Sep 2026
- 2026LiveProBench: Can Streaming Video Models Really Interact Like Humans?11 Sep 2026
- 2026How Well Can Your Video Model Remember? Measuring Memory-Budget Trade-offs in Long Video Understanding17 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2026OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interaction
- 2026StreamReady: Learning What to Answer and When in Long Streaming Videos
- 2026STRIDE: When to Speak Meets Sequence Denoising for Streaming Video Understanding
- 2026LiveStarPro: Proactive Streaming Video Understanding with Hierarchical Memory for Long-Horizon Streams
- 2026StreamOPD: A Post-Training Recipe with Spatio-Temporal Cue Gating for Streaming Video Understanding
- 2026What Should a Streaming Video Model Remember?
- 2025StreamGaze: Gaze-Guided Temporal Reasoning and Proactive Understanding in Streaming Videos
- 2026Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously