mlsanityData to 7 Oct 2026

StreamPro: From Reactive Perception to Proactive Decision-Making in Streaming Video

arXiv 2605.16381 ↗First posted 11 May 2026cs.CV cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Proactive streaming video understanding requires models to continuously process video streams and decide when to respond, rather than merely what to respond. This naturally introduces a decision-making problem under partial observations, where models must balance early prediction against sufficient evidence. However, existing benchmarks largely follow a "see-then-answer" paradigm, where responses are triggered only after explicit evidence appears, effectively reducing proactive reasoning to delayed perception. As a result, they fail to evaluate a model's ability to make timely and reliable decisions under incomplete observations. Moreover, training proactive models is inherently challenging due to the extreme imbalance between silence and response signals in streaming trajectories, as well as the need to jointly optimize response correctness and timing. To address these challenges, we introduce StreamPro-Bench, a new benchmark that evaluates streaming models from three complementary perspectives: Perception Understanding, Temporal Reasoning, and Proactive Agency, where the last measures a model's ability to make early yet reliable decisions under partial observations. We further propose StreamPro, a two-stage training framework for proactive learning. First, we introduce CB-Stream Loss to mitigate the severe supervision imbalance during supervised fine-tuning (SFT). Then, we apply Group Relative Policy Optimization (GRPO) with a multi-grained reward design that involves both turn-level and trajectory-level rewards. Experiments show that StreamPro significantly improves proactive performance. On StreamPro-Bench, it achieves 41.5, substantially outperforming the previous best (10.4), while also maintaining strong performance on real-time streaming benchmarks, achieving 78.9 on StreamingBench-RTVU.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 39 of its 45 references that are in the map. First posted on arXiv 11 May 2026.

Builds on

39 of 45 references.

References · 39

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  3. 2025Qwen2.5-VL Technical Reportcited by 5,418
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  5. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  6. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  7. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  8. 2025Qwen3-VL Technical Reportcited by 2,534
  9. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  10. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  11. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
  12. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,239
27 more
  1. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
  2. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 644
  3. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 550
  4. 2024VideoLLM-online: Online Video Large Language Model for Streaming Videocited by 216
  5. 2024VideoChat-Flash: Hierarchical Compression for Long-Context Video Modelingcited by 212
  6. 2024StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understandingcited by 135
  7. 2025OVO-Bench: How Far is Your Video-LLMs from Real-World Online Video Understanding?cited by 134
  8. 2025Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reactioncited by 122
  9. 2025TimeChat-Online: 80% Visual Tokens are Naturally Redundant in Streaming Videoscited by 90
  10. 2025StreamingVLM: Real-Time Understanding for Infinite Video Streamscited by 89
  11. 2025Flash-VStream: Efficient Real-Time Understanding for Long Video Streamscited by 78
  12. 2025StreamForest: Efficient Online Video Understanding with Persistent Event Memorycited by 71
  13. 2024E.T. Bench: Towards Open-Ended Event-Level Video-Language Understandingcited by 68
  14. 2025StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistantcited by 56
  15. 2024Advancing Video Anomaly Detection: A Concise Review and a New Datasetcited by 48
  16. 2025OmniMMI: A Comprehensive Multi-modal Interaction Benchmark in Streaming Video Contextscited by 48
  17. 2024VideoLLM Knows When to Speak: Enhancing Time-Sensitive Video Comprehension with Video-Text Duet Interaction Formatcited by 44
  18. 2025StreamMind: Unlocking Full Frame Rate Streaming Video Dialogue through Event-Gated Cognitioncited by 44
  19. 2025ViSpeak: Visual Instruction Feedback in Streaming Videoscited by 38
  20. 2025StreamAgent: Towards Anticipatory Agents for Streaming Video Understandingcited by 29
  21. 2025Eyes Wide Open: Ego Proactive Video-LLM for Streaming Videocited by 25
  22. 2025Streaming Video Instruction Tuningcited by 25
  23. 2025ProactiveVideoQA: A Comprehensive Benchmark Evaluating Proactive Interactions in Video Large Language Modelscited by 24
  24. 2025MMDuet2: Enhancing Proactive Interaction of Video MLLMs with Multi-Turn Reinforcement Learningcited by 18
  25. 2026AURA: Always-On Understanding and Real-Time Assistance via Video Streamscited by 15
  26. 2026Thinking in Streaming Videocited by 14
  27. 2026Progressive Online Video Understanding with Evidence-Aligned Timing and Transparent Decisionscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interaction
  2. 2026StreamReady: Learning What to Answer and When in Long Streaming Videos
  3. 2026STRIDE: When to Speak Meets Sequence Denoising for Streaming Video Understanding
  4. 2026LiveStarPro: Proactive Streaming Video Understanding with Hierarchical Memory for Long-Horizon Streams
  5. 2026StreamOPD: A Post-Training Recipe with Spatio-Temporal Cue Gating for Streaming Video Understanding
  6. 2026What Should a Streaming Video Model Remember?
  7. 2025StreamGaze: Gaze-Guided Temporal Reasoning and Proactive Understanding in Streaming Videos
  8. 2026Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously