Progressive Online Video Understanding with Evidence-Aligned Timing and Transparent Decisions
Not on the map. Why?
Abstract (arXiv)
Visual agents operating in the wild must respond to queries precisely when sufficient evidence first appears in a video stream, a critical capability that is overlooked by conventional video LLMs evaluated in offline settings. The shift to an online, streaming paradigm introduces significant challenges: a lack of decision transparency, the difficulty of aligning response timing with visual evidence, and the need to maintain a global, causally consistent understanding under tight computational budgets. To address these issues, we propose a novel framework that decouples reasoning control from memory integration. We introduce textbfmodel, an instantiation of this framework with two core components. First, the Active Thinking Decision Maker (ATDM) is a transparent reasoning controller that externalizes its decision process using observable progress (boldsymbolρ) and confidence (boldsymbolc) metrics. This allows it to precisely time its response tr to match the first-sufficient-evidence timestamp t^star while streaming its reasoning to the user. Second, the Hierarchical Progressive Semantic Integration (HPSI) module acts as an efficient memory system. It employs a set of learnable, multi-level aggregation tokens that are propagated across clips to build a rich, global cognitive state without exceeding token budgets. %Our approach sets a new standard on key online video understanding benchmarks, achieving strong performance of 71.6% on StreamingBench and 46.9% on OVOBench, demonstrating a robust solution for evidence-aligned and transparent online video analysis. Extensive experiments demonstrate the effectiveness of ATDM and HPSI, e.g., Thinking-QwenVL improves the accuracy of the previous state-of-the-art from 67.63% to 71.60% on the StreamingBench benchmark.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 35 of its 44 references that are in the map. First posted on arXiv 20 Apr 2026.
Builds on
35 of 44 references.
References · 35
- 2023Visual Instruction Tuningcited by 7,792
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
- 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
- 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
- 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 749
- 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 644
- 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Modelscited by 539
- 2024Long Context Transfer from Language to Visioncited by 502
23 more
- 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 502
- 2024LVBench: An Extreme Long Video Understanding Benchmarkcited by 402
- 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understandingcited by 399
- 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 386
- 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understandingcited by 341
- 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videoscited by 303
- 2024VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videoscited by 242
- 2024VideoLLM-online: Online Video Large Language Model for Streaming Videocited by 216
- 2024Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understandingcited by 210
- 2024Streaming Long Video Understanding with Large Language Modelscited by 164
- 2024Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streamscited by 160
- 2024StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understandingcited by 135
- 2024Video-RAG: Visually-aligned Retrieval-Augmented Long Video Comprehensioncited by 134
- 2025OVO-Bench: How Far is Your Video-LLMs from Real-World Online Video Understanding?cited by 134
- 2024Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Inputcited by 132
- 2025Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reactioncited by 122
- 2024Video-CCAM: Enhancing Video-Language Understanding with Causal Cross-Attention Masks for Short and Long Videoscited by 95
- 2025TimeChat-Online: 80% Visual Tokens are Naturally Redundant in Streaming Videoscited by 90
- 2024Apollo: An Exploration of Video Understanding in Large Multimodal Modelscited by 87
- 2025Streaming Video Understanding and Multi-round Interaction with Memory-enhanced Knowledgecited by 68
- 2025StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistantcited by 56
- 2025RTV-Bench: Benchmarking MLLM Continuous Perception, Understanding and Reasoning through Real-Time Videocited by 21
- 2025Confidence-guided Refinement Reasoning for Zero-shot Question Answeringcited by 3
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026Dynamic Hub-and-Spoke Memory for Streaming Video Understandingcited by 4
- 2026StreamPro: From Reactive Perception to Proactive Decision-Making in Streaming Videocited by 3
- 2026Omni-Streaming Thinkingcited by 1
- 2026InterOCF: Spatio-Temporal 2D-3D Interaction for Camera-Only 4D Occupancy Forecastingcited by 0
- 2026Agentic Video Understanding: A Surveycited by 0
- 2026Agentic Video Understanding: A Survey21 Sep 2026
- 2026Omni-Streaming Thinking14 Sep 2026
- 2026Dynamic Hub-and-Spoke Memory for Streaming Video Understanding31 Aug 2026
- 2026InterOCF: Spatio-Temporal 2D-3D Interaction for Camera-Only 4D Occupancy Forecasting27 Jul 2026
- 2026StreamPro: From Reactive Perception to Proactive Decision-Making in Streaming Video11 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026Thinking in Streaming Video
- 2026Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously
- 2025Active Video Perception: Iterative Evidence Seeking for Agentic Long Video Understanding
- 2025Select Less, Reason More: Prioritizing Evidence Purity for Video Reasoning
- 2026Perception Before Reasoning: Dynamic Latent Reasoning for Video Understanding and Question Answering
- 2026EventMemAgent: Hierarchical Event-Centric Memory for Online Video Understanding with Adaptive Tool Use
- 2026Reinforce to Learn, Elect to Reason: A Dual Paradigm for Video Reasoning
- 2025When Thinking Drifts: Evidential Grounding for Robust Video Reasoning