Think While Watching: Online Streaming Segment-Level Memory for Multi-Turn Video Reasoning in Multimodal Large Language Models
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Multimodal large language models (MLLMs) have shown strong performance on offline video understanding, but most are limited to offline inference or have weak online reasoning, making multi-turn interaction over continuously arriving video streams difficult. Existing streaming methods typically use an interleaved perception-generation paradigm, which prevents concurrent perception and generation and leads to early memory decay as streams grow, hurting long-range dependency modeling. We propose Think While Watching, a memory-anchored streaming video reasoning framework that preserves continuous segment-level memory during multi-turn interaction. We build a three-stage, multi-round chain-of-thought dataset and adopt a stage-matched training strategy, while enforcing strict causality through a segment-level streaming causal mask and streaming positional encoding. During inference, we introduce an efficient pipeline that overlaps watching and thinking and adaptively selects the best attention backend. Under both single-round and multi-round streaming input protocols, our method achieves strong results. Built on Qwen3-VL, it improves single-round accuracy by 2.6% on StreamingBench and by 3.79% on OVO-Bench. In the multi-round setting, it maintains performance while reducing output tokens by 56%. Code is available at: https://github.com/wl666hhh/Think_While_Watching/
Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 33 of its 45 references that are in the map. First posted on arXiv 12 Mar 2026.
Builds on
33 of 45 references.
Closest ancestors · 8
Same direction, newest first
- 2025StreamGaze: Gaze-Guided Temporal Reasoning and Proactive Understanding in Streaming Videos1 Dec
- 2025Thinking with Video: Video Generation as a Promising Multimodal Reasoning Paradigm6 Nov
- 2025StreamingCoT: A Dataset for Temporal Dynamics and Multimodal Chain-of-Thought Reasoning in Streaming VideoQA29 Oct
- 2025Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoning6 Aug
- 2025Scaling RL to Long Videos10 Jul
- 2025RTV-Bench: Benchmarking MLLM Continuous Perception, Understanding and Reasoning through Real-Time Video4 May
- 2025Video-R1: Reinforcing Video Reasoning in MLLMs27 Mar
- 2024Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognition7 May
Other references · 25
- 2024GPT-4o System Cardcited by 4,467
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
- 2024LVBench: An Extreme Long Video Understanding Benchmarkcited by 402
- 2021Self-attention Does Not Need O(n2) Memorycited by 152
- 2024StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understandingcited by 135
- 2025TimeChat-Online: 80% Visual Tokens are Naturally Redundant in Streaming Videoscited by 90
- 2025StreamingVLM: Real-Time Understanding for Infinite Video Streamscited by 89
- 2025Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memorycited by 88
- 2025Flash-VStream: Efficient Real-Time Understanding for Long Video Streamscited by 78
- 2025StreamForest: Efficient Online Video Understanding with Persistent Event Memorycited by 71
13 more
- 2025LiveVLM: Efficient Online Video Understanding via Streaming-Oriented KV Cache and Retrievalcited by 50
- 2025StreamingTOM: Streaming Token Compression for Efficient Video Understandingcited by 41
- 2025ViSpeak: Visual Instruction Feedback in Streaming Videoscited by 38
- 2025StreamAgent: Towards Anticipatory Agents for Streaming Video Understandingcited by 29
- 2025Eyes Wide Open: Ego Proactive Video-LLM for Streaming Videocited by 25
- 2025StreamKV: Streaming Video Question-Answering with Segment-based KV Cache Retrieval and Compressioncited by 24
- 2024StreamChat: Chatting with Streaming Videocited by 23
- 2025LiveStar: Live Streaming Assistant for Real-World Online Video Understandingcited by 22
- 2025StreamingThinker: Large Language Models Can Think While Readingcited by 15
- 2025StreamEQA: Towards Streaming Video Understanding for Embodied Scenarioscited by 11
- 2026Speak While Watching: Unleashing TRUE Real-Time Video Understanding Capability of Multimodal Large Language Modelscited by 7
- 2025CacheFlow: Fast Human Motion Prediction by Cached Normalizing Flowcited by 4
- 2025V-Rex: Real-Time Streaming Video LLM Acceleration via Dynamic KV Cache Retrievalcited by 4
Built on by
Citing papers per year: 2026: 7.
7 papers.
- 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
- 2026Look Light, Think Heavy: What Multimodal Chain-of-Thought Reasoning Can and Cannot Docited by 3
- 2026FOLIO: Focused Semantic Memory for Streaming Video Understandingcited by 3
- 2026Omni-Streaming Thinkingcited by 1
- 2026TimeInteract: Towards Real-Time Interactive Intelligence for Streaming Time Seriescited by 0
- 2026APM-Bench: Benchmarking Cross-session Persistent Memory for Egocentric Streaming Video Assistantscited by 0
- 2026OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interactioncited by 0
- 2026OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interaction1 Oct 2026
- 2026APM-Bench: Benchmarking Cross-session Persistent Memory for Egocentric Streaming Video Assistants29 Sep 2026
- 2026TimeInteract: Towards Real-Time Interactive Intelligence for Streaming Time Series22 Sep 2026
- 2026Omni-Streaming Thinking14 Sep 2026
- 2026FOLIO: Focused Semantic Memory for Streaming Video Understanding14 Jul 2026
- 2026Look Light, Think Heavy: What Multimodal Chain-of-Thought Reasoning Can and Cannot Do21 Jun 2026
- 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Application10 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2026Thinking in Streaming Video
- 2026Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously
- 2026WAT: Online Video Understanding Needs Watching Before Thinking
- 2026Think-as-You-See: Streaming Chain-of-Thought Reasoning for Large Vision-Language Models
- 2026Watch, Remember, Reason: Human-View Video Understanding with MLLMs
- 2026Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understanding
- 2026Towards a Dynamic and Fixed-budget Memory Bank for Efficient Streaming Video Understanding
- 2025SVBench: A Benchmark with Temporal Multi-Turn Dialogues for Streaming Video Understanding