Watch-Think-Interact: Bootstrapping Long-Horizon Multi-Turn Streaming Video Reasoning with Reinforcement Learning
Not on the map. Why?
Abstract (arXiv)
Streaming video assistance requires models to answer asynchronous questions from an observed prefix under a fixed context budget. Existing approaches model response timing or compress history, but an online state formed before future questions are known can omit visual details before later questions reveal their relevance; the retained state alone cannot recover them. We introduce Watch-Think-Interact (WTI), a closed-loop framework for multi-question streaming video reasoning. WTI maintains compact natural-language memory entries tagged with source-video time ranges; these entries support direct reasoning when sufficient and otherwise anchor selective recall of finer visual evidence. For each question, WTI answers when current context and memory suffice, continues watching when required evidence has not appeared, or recalls a relevant past interval and decides again after incorporating the returned chunks, without replaying the full observed history. To train this behavior, we construct WTI-82K, comprising 82,335 timed questions across 4,812 causally aligned trajectories, and develop Stream-GDPO to optimize complete multi-question streaming rollouts using trajectory-level feedback for response timing, source-video recall, and memory updates. WTI achieves state-of-the-art aggregate performance among the compared open-source streaming baselines, reaching 83.3% on StreamingBench and 73.6% weighted overall accuracy on OVO-Bench.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 28 of its 37 references that are in the map. First posted on arXiv 29 Sep 2026.
Builds on
28 of 37 references.
References · 28
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
- 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
- 2025Group Sequence Policy Optimizationcited by 835
- 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 665
- 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
- 2024Long Context Transfer from Language to Visioncited by 515
- 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understandingcited by 356
- 2025MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agentcited by 246
- 2024VideoLLM-online: Online Video Large Language Model for Streaming Videocited by 225
16 more
- 2024VideoChat-Flash: Hierarchical Compression for Long-Context Video Modelingcited by 220
- 2026GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimizationcited by 193
- 2025OVO-Bench: How Far is Your Video-LLMs from Real-World Online Video Understanding?cited by 142
- 2025Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reactioncited by 127
- 2025StreamingVLM: Real-Time Understanding for Infinite Video Streamscited by 95
- 2025Flash-VStream: Efficient Real-Time Understanding for Long Video Streamscited by 81
- 2025StreamForest: Efficient Online Video Understanding with Persistent Event Memorycited by 78
- 2026MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interactioncited by 78
- 2025StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistantcited by 63
- 2024Online Video Understanding: OVBench and VideoChat-Onlinecited by 61
- 2025SVBench: A Benchmark with Temporal Multi-Turn Dialogues for Streaming Video Understandingcited by 57
- 2025ViSpeak: Visual Instruction Feedback in Streaming Videoscited by 42
- 2025Look Back to Reason Forward: Revisitable Memory for Long-Context LLM Agentscited by 38
- 2025LiveStar: Live Streaming Assistant for Real-World Online Video Understandingcited by 25
- 2026Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneouslycited by 22
- 2026Explore with Long-term Memory: A Benchmark and Multimodal LLM-based Reinforcement Learning Framework for Embodied Explorationcited by 13
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Thinking in Streaming Video
- 2026Think While Watching: Online Streaming Segment-Level Memory for Multi-Turn Video Reasoning in Multimodal Large Language Models
- 2026StreamScout: Learning When to Look Deeper for Streaming Video Understanding
- 2026StreamReady: Learning What to Answer and When in Long Streaming Videos
- 2026VAMR: Multi-Question Agentic Reasoning for Efficient Long-Form Video Understanding
- 2025StreamingCoT: A Dataset for Temporal Dynamics and Multimodal Chain-of-Thought Reasoning in Streaming VideoQA
- 2026Sprout: Building Dynamic Memory While Reasoning for Agentic Video Understanding
- 2026Perception Before Reasoning: Dynamic Latent Reasoning for Video Understanding and Question Answering