mlsanityData to 7 Oct 2026

Progressive Online Video Understanding with Evidence-Aligned Timing and Transparent Decisions

arXiv 2604.18459 ↗First posted 20 Apr 2026cs.CV cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Visual agents operating in the wild must respond to queries precisely when sufficient evidence first appears in a video stream, a critical capability that is overlooked by conventional video LLMs evaluated in offline settings. The shift to an online, streaming paradigm introduces significant challenges: a lack of decision transparency, the difficulty of aligning response timing with visual evidence, and the need to maintain a global, causally consistent understanding under tight computational budgets. To address these issues, we propose a novel framework that decouples reasoning control from memory integration. We introduce textbfmodel, an instantiation of this framework with two core components. First, the Active Thinking Decision Maker (ATDM) is a transparent reasoning controller that externalizes its decision process using observable progress (boldsymbolρ) and confidence (boldsymbolc) metrics. This allows it to precisely time its response tr to match the first-sufficient-evidence timestamp t^star while streaming its reasoning to the user. Second, the Hierarchical Progressive Semantic Integration (HPSI) module acts as an efficient memory system. It employs a set of learnable, multi-level aggregation tokens that are propagated across clips to build a rich, global cognitive state without exceeding token budgets. %Our approach sets a new standard on key online video understanding benchmarks, achieving strong performance of 71.6% on StreamingBench and 46.9% on OVOBench, demonstrating a robust solution for evidence-aligned and transparent online video analysis. Extensive experiments demonstrate the effectiveness of ATDM and HPSI, e.g., Thinking-QwenVL improves the accuracy of the previous state-of-the-art from 67.63% to 71.60% on the StreamingBench benchmark.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 35 of its 44 references that are in the map. First posted on arXiv 20 Apr 2026.

Builds on

35 of 44 references.

References · 35

  1. 2023Visual Instruction Tuningcited by 7,792
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  3. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  4. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  5. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  6. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
  7. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
  8. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
  9. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 749
  10. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 644
  11. 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Modelscited by 539
  12. 2024Long Context Transfer from Language to Visioncited by 502
23 more
  1. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 502
  2. 2024LVBench: An Extreme Long Video Understanding Benchmarkcited by 402
  3. 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understandingcited by 399
  4. 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 386
  5. 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understandingcited by 341
  6. 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videoscited by 303
  7. 2024VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videoscited by 242
  8. 2024VideoLLM-online: Online Video Large Language Model for Streaming Videocited by 216
  9. 2024Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understandingcited by 210
  10. 2024Streaming Long Video Understanding with Large Language Modelscited by 164
  11. 2024Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streamscited by 160
  12. 2024StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understandingcited by 135
  13. 2024Video-RAG: Visually-aligned Retrieval-Augmented Long Video Comprehensioncited by 134
  14. 2025OVO-Bench: How Far is Your Video-LLMs from Real-World Online Video Understanding?cited by 134
  15. 2024Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Inputcited by 132
  16. 2025Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reactioncited by 122
  17. 2024Video-CCAM: Enhancing Video-Language Understanding with Causal Cross-Attention Masks for Short and Long Videoscited by 95
  18. 2025TimeChat-Online: 80% Visual Tokens are Naturally Redundant in Streaming Videoscited by 90
  19. 2024Apollo: An Exploration of Video Understanding in Large Multimodal Modelscited by 87
  20. 2025Streaming Video Understanding and Multi-round Interaction with Memory-enhanced Knowledgecited by 68
  21. 2025StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistantcited by 56
  22. 2025RTV-Bench: Benchmarking MLLM Continuous Perception, Understanding and Reasoning through Real-Time Videocited by 21
  23. 2025Confidence-guided Refinement Reasoning for Zero-shot Question Answeringcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Thinking in Streaming Video
  2. 2026Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously
  3. 2025Active Video Perception: Iterative Evidence Seeking for Agentic Long Video Understanding
  4. 2025Select Less, Reason More: Prioritizing Evidence Purity for Video Reasoning
  5. 2026Perception Before Reasoning: Dynamic Latent Reasoning for Video Understanding and Question Answering
  6. 2026EventMemAgent: Hierarchical Event-Centric Memory for Online Video Understanding with Adaptive Tool Use
  7. 2026Reinforce to Learn, Elect to Reason: A Dual Paradigm for Video Reasoning
  8. 2025When Thinking Drifts: Evidential Grounding for Robust Video Reasoning