mlsanityData to 7 Oct 2026

Think While Watching: Online Streaming Segment-Level Memory for Multi-Turn Video Reasoning in Multimodal Large Language Models

arXiv 2603.11896 ↗First posted 12 Mar 2026cs.CV cs.AI cs.CLcited by 7

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Multimodal large language models (MLLMs) have shown strong performance on offline video understanding, but most are limited to offline inference or have weak online reasoning, making multi-turn interaction over continuously arriving video streams difficult. Existing streaming methods typically use an interleaved perception-generation paradigm, which prevents concurrent perception and generation and leads to early memory decay as streams grow, hurting long-range dependency modeling. We propose Think While Watching, a memory-anchored streaming video reasoning framework that preserves continuous segment-level memory during multi-turn interaction. We build a three-stage, multi-round chain-of-thought dataset and adopt a stage-matched training strategy, while enforcing strict causality through a segment-level streaming causal mask and streaming positional encoding. During inference, we introduce an efficient pipeline that overlaps watching and thinking and adaptively selects the best attention backend. Under both single-round and multi-round streaming input protocols, our method achieves strong results. Built on Qwen3-VL, it improves single-round accuracy by 2.6% on StreamingBench and by 3.79% on OVO-Bench. In the multi-round setting, it maintains performance while reducing output tokens by 56%. Code is available at: https://github.com/wl666hhh/Think_While_Watching/

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 33 of its 45 references that are in the map. First posted on arXiv 12 Mar 2026.

Builds on

33 of 45 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025StreamGaze: Gaze-Guided Temporal Reasoning and Proactive Understanding in Streaming Videos1 Dec
  2. 2025Thinking with Video: Video Generation as a Promising Multimodal Reasoning Paradigm6 Nov
  3. 2025StreamingCoT: A Dataset for Temporal Dynamics and Multimodal Chain-of-Thought Reasoning in Streaming VideoQA29 Oct
  4. 2025Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoning6 Aug
  5. 2025Scaling RL to Long Videos10 Jul
  6. 2025RTV-Bench: Benchmarking MLLM Continuous Perception, Understanding and Reasoning through Real-Time Video4 May
  7. 2025Video-R1: Reinforcing Video Reasoning in MLLMs27 Mar
  8. 2024Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognition7 May

Other references · 25

  1. 2024GPT-4o System Cardcited by 4,467
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  3. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  4. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  5. 2024LVBench: An Extreme Long Video Understanding Benchmarkcited by 402
  6. 2021Self-attention Does Not Need O(n2) Memorycited by 152
  7. 2024StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understandingcited by 135
  8. 2025TimeChat-Online: 80% Visual Tokens are Naturally Redundant in Streaming Videoscited by 90
  9. 2025StreamingVLM: Real-Time Understanding for Infinite Video Streamscited by 89
  10. 2025Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memorycited by 88
  11. 2025Flash-VStream: Efficient Real-Time Understanding for Long Video Streamscited by 78
  12. 2025StreamForest: Efficient Online Video Understanding with Persistent Event Memorycited by 71
13 more
  1. 2025LiveVLM: Efficient Online Video Understanding via Streaming-Oriented KV Cache and Retrievalcited by 50
  2. 2025StreamingTOM: Streaming Token Compression for Efficient Video Understandingcited by 41
  3. 2025ViSpeak: Visual Instruction Feedback in Streaming Videoscited by 38
  4. 2025StreamAgent: Towards Anticipatory Agents for Streaming Video Understandingcited by 29
  5. 2025Eyes Wide Open: Ego Proactive Video-LLM for Streaming Videocited by 25
  6. 2025StreamKV: Streaming Video Question-Answering with Segment-based KV Cache Retrieval and Compressioncited by 24
  7. 2024StreamChat: Chatting with Streaming Videocited by 23
  8. 2025LiveStar: Live Streaming Assistant for Real-World Online Video Understandingcited by 22
  9. 2025StreamingThinker: Large Language Models Can Think While Readingcited by 15
  10. 2025StreamEQA: Towards Streaming Video Understanding for Embodied Scenarioscited by 11
  11. 2026Speak While Watching: Unleashing TRUE Real-Time Video Understanding Capability of Multimodal Large Language Modelscited by 7
  12. 2025CacheFlow: Fast Human Motion Prediction by Cached Normalizing Flowcited by 4
  13. 2025V-Rex: Real-Time Streaming Video LLM Acceleration via Dynamic KV Cache Retrievalcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 7.

7 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Thinking in Streaming Video
  2. 2026Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously
  3. 2026WAT: Online Video Understanding Needs Watching Before Thinking
  4. 2026Think-as-You-See: Streaming Chain-of-Thought Reasoning for Large Vision-Language Models
  5. 2026Watch, Remember, Reason: Human-View Video Understanding with MLLMs
  6. 2026Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understanding
  7. 2026Towards a Dynamic and Fixed-budget Memory Bank for Efficient Streaming Video Understanding
  8. 2025SVBench: A Benchmark with Temporal Multi-Turn Dialogues for Streaming Video Understanding