StreamReady: Learning What to Answer and When in Long Streaming Videos
Not on the map. Why?
Abstract (arXiv)
Streaming video understanding often involves time-sensitive scenarios where models need to answer exactly when the supporting visual evidence appears: answering before the evidence reflects speculation, answering after it has passed reduces real-time utility. To capture this behavior, we introduce a readiness-aware formulation of streaming video understanding with the Answer Readiness Score (ARS), a timing-aware objective with asymmetric early and late penalties. When combined with correctness, ARS defines an effective accuracy that measures not just whether a model is right, but whether it answers at the appropriate moment. Building on this formulation, we introduce StreamReady, a framework to unify temporal reasoning with on-time answering through a lightweight readiness mechanism that decides if sufficient evidence has been observed before responding. To evaluate this capability, we further introduce ProReady-QA, a benchmark with annotated answer evidence windows and proactive multi-turn questions across local and global contexts. StreamReady achieves superior performance on ProReady-QA, and consistently outperforms prior methods across eight additional streaming and offline long-video benchmarks, demonstrating robust and broadly generalizable video understanding capability.
Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 56 of its 61 references that are in the map. First posted on arXiv 9 Mar 2026.
Builds on
56 of 61 references.
References · 56
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
- 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
- 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
- 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
- 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
- 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 666
- 2019ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answeringcited by 559
38 more
- 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Modelscited by 539
- 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 536
- 2024Long Context Transfer from Language to Visioncited by 502
- 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understandingcited by 399
- 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understandingcited by 341
- 2024VideoMamba: State Space Model for Efficient Video Understandingcited by 325
- 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 323
- 2024VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videoscited by 242
- 2024MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understandingcited by 223
- 2024VideoLLM-online: Online Video Large Language Model for Streaming Videocited by 216
- 2024VideoChat-Flash: Hierarchical Compression for Long-Context Video Modelingcited by 212
- 2020MovieNet: A Holistic Dataset for Movie Understandingcited by 206
- 2025VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interactioncited by 191
- 2024Streaming Long Video Understanding with Large Language Modelscited by 164
- 2024Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streamscited by 160
- 2024StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understandingcited by 135
- 2025OVO-Bench: How Far is Your Video-LLMs from Real-World Online Video Understanding?cited by 134
- 2024LITA: Language Instructed Temporal-Localization Assistantcited by 124
- 2025Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reactioncited by 122
- 2023VideoLLM: Modeling Video Sequence with Large Language Modelscited by 116
- 2025Streaming Video Question-Answering with In-context Video KV-Cache Retrievalcited by 112
- 2025TimeChat-Online: 80% Visual Tokens are Naturally Redundant in Streaming Videoscited by 90
- 2024MovieChat+: Question-aware Sparse Memory for Long Video Question Answeringcited by 73
- 2025StreamForest: Efficient Online Video Understanding with Persistent Event Memorycited by 71
- 2025Streaming Video Understanding and Multi-round Interaction with Memory-enhanced Knowledgecited by 68
- 2024Online Video Understanding: OVBench and VideoChat-Onlinecited by 58
- 2025StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistantcited by 56
- 2025InfiniPot-V: Memory-Constrained KV Cache Compression for Streaming Video Understandingcited by 54
- 2025StreamMem: Query-Agnostic KV Cache Memory for Streaming Video Understandingcited by 53
- 2025OmniMMI: A Comprehensive Multi-modal Interaction Benchmark in Streaming Video Contextscited by 48
- 2024Memory Consolidation Enables Long-Context Video Understandingcited by 45
- 2024VideoLLM Knows When to Speak: Enhancing Time-Sensitive Video Comprehension with Video-Text Duet Interaction Formatcited by 44
- 2025ViSpeak: Visual Instruction Feedback in Streaming Videoscited by 38
- 2025VideoAgent2: Enhancing the LLM-Based Agent System for Long-Form Video Understanding by Uncertainty-Aware CoTcited by 36
- 2024From Seconds to Hours: Reviewing MultiModal Large Language Models on Comprehensive Long Video Understandingcited by 29
- 2025StreamAgent: Towards Anticipatory Agents for Streaming Video Understandingcited by 29
- 2025ProactiveVideoQA: A Comprehensive Benchmark Evaluating Proactive Interactions in Video Large Language Modelscited by 24
- 2023Probing Conceptual Understanding of Large Visual-Language Modelscited by 17
50 most cited of 56 All 56 (JSON)
Built on by
Citing papers per year: 2026: 18.
18 papers.
Sort:
By area:
- 2026A Simple Baseline for Streaming Video Understandingcited by 29
- 2026StreamOV: Streaming Omni-Video Understanding via Evidence-Guided Memory and Response Triggeringcited by 4
- 2026IPIBench: Evaluating Interactive Proactive Intelligence of MLLMs under Continuous Streamscited by 4
- 2026StreamTTT: Reconciling Real-Time Perception and Long-Term Memory in Streaming VLMscited by 3
- 2026StreamScout: Learning When to Look Deeper for Streaming Video Understandingcited by 2
- 2026ShallowStream: Index Shallow then Answer Deep for Streaming Video Understandingcited by 2
- 2026Don't Pause: Streaming Video-Language Synchrony for Online Video Understandingcited by 1
- 2026GROVE: Growing and Reasoning over Temporally Stratified Memory from Streaming Video Experiencecited by 1
- 2026Omni-Streaming Thinkingcited by 1
- 2026VISTA: Video Interaction Spatio-Temporal Analysis Benchmarkcited by 0
- 2026Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understandingcited by 0
- 2026Companion-style QA Assistance in Ego-Visioncited by 0
6 more
- 2026LiveProBench: Can Streaming Video Models Really Interact Like Humans?cited by 0
- 2026Live Assistant: Learning Whether, When, and Whom to Assist in Real-World Live Social Streamscited by 0
- 2026STORM-Bench: Evaluating Online Video QA under Evolving and Incomplete Evidencecited by 0
- 2026Agentic Video Understanding: A Surveycited by 0
- 2026FlashBack: Knowing When to Remember in Streaming Vision-Language Modelscited by 0
- 2026OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interactioncited by 0
- 2026OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interaction1 Oct 2026
- 2026FlashBack: Knowing When to Remember in Streaming Vision-Language Models1 Oct 2026
- 2026STORM-Bench: Evaluating Online Video QA under Evolving and Incomplete Evidence25 Sep 2026
- 2026Live Assistant: Learning Whether, When, and Whom to Assist in Real-World Live Social Streams23 Sep 2026
- 2026Agentic Video Understanding: A Survey21 Sep 2026
- 2026Omni-Streaming Thinking14 Sep 2026
- 2026LiveProBench: Can Streaming Video Models Really Interact Like Humans?11 Sep 2026
- 2026Companion-style QA Assistance in Ego-Vision6 Sep 2026
- 2026Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understanding3 Sep 2026
- 2026ShallowStream: Index Shallow then Answer Deep for Streaming Video Understanding2 Sep 2026
- 2026StreamScout: Learning When to Look Deeper for Streaming Video Understanding31 Aug 2026
- 2026StreamTTT: Reconciling Real-Time Perception and Long-Term Memory in Streaming VLMs13 Aug 2026
6 more
- 2026GROVE: Growing and Reasoning over Temporally Stratified Memory from Streaming Video Experience3 Aug 2026
- 2026Don't Pause: Streaming Video-Language Synchrony for Online Video Understanding5 Jun 2026
- 2026IPIBench: Evaluating Interactive Proactive Intelligence of MLLMs under Continuous Streams26 May 2026
- 2026StreamOV: Streaming Omni-Video Understanding via Evidence-Guided Memory and Response Triggering25 May 2026
- 2026VISTA: Video Interaction Spatio-Temporal Analysis Benchmark2 May 2026
- 2026A Simple Baseline for Streaming Video Understanding2 Apr 2026
- 2026IPIBench: Evaluating Interactive Proactive Intelligence of MLLMs under Continuous Streamscited by 4
- 2026VISTA: Video Interaction Spatio-Temporal Analysis Benchmarkcited by 0
- 2026Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understandingcited by 0
- 2026FlashBack: Knowing When to Remember in Streaming Vision-Language Modelscited by 0
- 2026FlashBack: Knowing When to Remember in Streaming Vision-Language Models1 Oct 2026
- 2026Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understanding3 Sep 2026
- 2026IPIBench: Evaluating Interactive Proactive Intelligence of MLLMs under Continuous Streams26 May 2026
- 2026VISTA: Video Interaction Spatio-Temporal Analysis Benchmark2 May 2026
- 2026Agentic Video Understanding: A Surveycited by 0
- 2026Agentic Video Understanding: A Survey21 Sep 2026
- 2026A Simple Baseline for Streaming Video Understandingcited by 29
- 2026StreamOV: Streaming Omni-Video Understanding via Evidence-Guided Memory and Response Triggeringcited by 4
- 2026StreamTTT: Reconciling Real-Time Perception and Long-Term Memory in Streaming VLMscited by 3
- 2026StreamScout: Learning When to Look Deeper for Streaming Video Understandingcited by 2
- 2026Don't Pause: Streaming Video-Language Synchrony for Online Video Understandingcited by 1
- 2026GROVE: Growing and Reasoning over Temporally Stratified Memory from Streaming Video Experiencecited by 1
- 2026Omni-Streaming Thinkingcited by 1
- 2026Companion-style QA Assistance in Ego-Visioncited by 0
- 2026LiveProBench: Can Streaming Video Models Really Interact Like Humans?cited by 0
- 2026Live Assistant: Learning Whether, When, and Whom to Assist in Real-World Live Social Streamscited by 0
- 2026STORM-Bench: Evaluating Online Video QA under Evolving and Incomplete Evidencecited by 0
- 2026OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interactioncited by 0
- 2026OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interaction1 Oct 2026
- 2026STORM-Bench: Evaluating Online Video QA under Evolving and Incomplete Evidence25 Sep 2026
- 2026Live Assistant: Learning Whether, When, and Whom to Assist in Real-World Live Social Streams23 Sep 2026
- 2026Omni-Streaming Thinking14 Sep 2026
- 2026LiveProBench: Can Streaming Video Models Really Interact Like Humans?11 Sep 2026
- 2026Companion-style QA Assistance in Ego-Vision6 Sep 2026
- 2026StreamScout: Learning When to Look Deeper for Streaming Video Understanding31 Aug 2026
- 2026StreamTTT: Reconciling Real-Time Perception and Long-Term Memory in Streaming VLMs13 Aug 2026
- 2026GROVE: Growing and Reasoning over Temporally Stratified Memory from Streaming Video Experience3 Aug 2026
- 2026Don't Pause: Streaming Video-Language Synchrony for Online Video Understanding5 Jun 2026
- 2026StreamOV: Streaming Omni-Video Understanding via Evidence-Guided Memory and Response Triggering25 May 2026
- 2026A Simple Baseline for Streaming Video Understanding2 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2025StreamingCoT: A Dataset for Temporal Dynamics and Multimodal Chain-of-Thought Reasoning in Streaming VideoQA
- 2026Thinking in Streaming Video
- 2025StreamEQA: Towards Streaming Video Understanding for Embodied Scenarios
- 2025SVBench: A Benchmark with Temporal Multi-Turn Dialogues for Streaming Video Understanding
- 2025Streaming Video Instruction Tuning
- 2026What Should a Streaming Video Model Remember?
- 2026Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously
- 2025StreamGaze: Gaze-Guided Temporal Reasoning and Proactive Understanding in Streaming Videos