Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Online Video Large Language Models (VideoLLMs) play a critical role in supporting responsive, real-time interaction. Existing methods focus on streaming perception, lacking a synchronized logical reasoning stream. However, directly applying test-time scaling methods incurs unacceptable response latency. To address this trade-off, we propose Video Streaming Thinking (VST), a novel paradigm for streaming video understanding. It supports a thinking while watching mechanism, which activates reasoning over incoming video clips during streaming. This design improves timely comprehension and coherent cognition while preserving real-time responsiveness by amortizing LLM reasoning latency over video playback. Furthermore, we introduce a comprehensive post-training pipeline that integrates VST-SFT, which structurally adapts the offline VideoLLM to causal streaming reasoning, and VST-RL, which provides end-to-end improvement through self-exploration in a multi-turn video interaction environment. Additionally, we devise an automated training-data synthesis pipeline that uses video knowledge graphs to generate high-quality streaming QA pairs, with an entity-relation grounded streaming Chain-of-Thought to enforce multi-evidence reasoning and sustained attention to the video stream. Extensive evaluations show that VST-7B performs strongly on online benchmarks, e.g. 79.5% on StreamingBench and 59.3% on OVO-Bench. Meanwhile, VST remains competitive on offline long-form or reasoning benchmarks. Compared with Video-R1, VST responds 15.7 times faster and achieves +5.4% improvement on VideoHolmes, demonstrating higher efficiency and strong generalization across diverse video understanding tasks. Code, data, and models will be released at https://github.com/1ranGuan/VST.
Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 58 of its 70 references that are in the map. First posted on arXiv 12 Mar 2026.
Builds on
58 of 70 references.
Closest ancestors · 11
Same direction, newest first
- 2026ThinkOmni: Lifting Textual Reasoning to Omni-modal Scenarios via Guidance Decoding26 Feb
- 2026Video-o3: Native Interleaved Clue Seeking for Long Video Multi-Hop Reasoning30 Jan
- 2025OneThinker: All-in-one Reasoning Model for Image and Video2 Dec
- 2025REVISOR: Beyond Textual Reflection, Towards Multimodal Introspective Reasoning in Long-Form Video Understanding17 Nov
- 2025VideoChat-R1.5: Visual Test-Time Scaling to Reinforce Multimodal Reasoning by Iterative Perception25 Sep
- 2025Shuffle-R1: Efficient RL framework for Multimodal Large Language Models via Data-centric Dynamic Shuffle7 Aug
- 2025Scaling RL to Long Videos10 Jul
- 2025Video-Holmes: Can MLLM Think Like Holmes for Complex Video Reasoning?27 May
- 2025VideoRFT: Incentivizing Video Reasoning Capability in MLLMs via Reinforced Fine-Tuning18 May
- 2025Video-R1: Reinforcing Video Reasoning in MLLMs27 Mar
- 2024VideoEspresso: A Large-Scale Chain-of-Thought Dataset for Fine-Grained Video Reasoning via Core Frame Selection22 Nov
Other references · 47
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
- 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
35 more
- 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 723
- 2023VILA: On Pre-training for Visual Language Modelscited by 692
- 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 665
- 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 660
- 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
- 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 542
- 2024Long Context Transfer from Language to Visioncited by 515
- 2023Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Modelscited by 357
- 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understandingcited by 356
- 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 353
- 2025MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agentcited by 246
- 2024VideoLLM-online: Online Video Large Language Model for Streaming Videocited by 225
- 2024Streaming Long Video Understanding with Large Language Modelscited by 167
- 2024StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understandingcited by 144
- 2025OVO-Bench: How Far is Your Video-LLMs from Real-World Online Video Understanding?cited by 142
- 2025Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reactioncited by 127
- 2025Streaming Video Question-Answering with In-context Video KV-Cache Retrievalcited by 117
- 2025TimeChat-Online: 80% Visual Tokens are Naturally Redundant in Streaming Videoscited by 97
- 2025StreamingVLM: Real-Time Understanding for Infinite Video Streamscited by 95
- 2025Flash-VStream: Efficient Real-Time Understanding for Long Video Streamscited by 81
- 2025StreamForest: Efficient Online Video Understanding with Persistent Event Memorycited by 78
- 2025StreamMem: Query-Agnostic KV Cache Memory for Streaming Video Understandingcited by 58
- 2025LiveVLM: Efficient Online Video Understanding via Streaming-Oriented KV Cache and Retrievalcited by 54
- 2022TransRAC: Encoding Multi-scale Temporal Correlation with Transformers for Repetitive Action Countingcited by 44
- 2024Mini-Monkey: Alleviating the Semantic Sawtooth Effect for Lightweight MLLMs via Complementary Image Pyramidcited by 26
- 2025Streaming Video Instruction Tuningcited by 26
- 2025MindDrive: A Vision-Language-Action Model for Autonomous Driving via Online Reinforcement Learningcited by 25
- 2025Unleashing Hour-Scale Video Training for Long Video-Language Understandingcited by 23
- 2024ReVisionLLM: Recursive Vision-Language Model for Temporal Grounding in Hour-Long Videoscited by 18
- 2025StreamingThinker: Large Language Models Can Think While Readingcited by 15
- 2026EventMemAgent: Hierarchical Event-Centric Memory for Online Video Understanding with Adaptive Tool Usecited by 12
- 2025Recurrent Attention-based Token Selection for Efficient Streaming Video-LLMscited by 9
- 2025Cook and Clean Together: Teaching Embodied Agents for Parallel Task Executioncited by 6
- 2025VKnowU: Evaluating Visual Knowledge Understanding in Multimodal LLMscited by 5
- 2025MSTAR: Box-free Multi-query Scene Text Retrieval with Attention Recyclingcited by 1
Built on by
Citing papers per year: 2026: 22.
22 papers.
Sort:
By area:
- 2026LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligencecited by 39
- 2026Towards Generalizable Robotic Manipulation in Dynamic Environmentscited by 20
- 2026JoyAI-VL-Interaction: Real-Time Vision-Language Interaction Intelligencecited by 14
- 2026Generation Models Know Space: Unleashing Implicit 3D Priors for Scene Understandingcited by 7
- 2026ObjectStream: Latent Objects as Memory Anchors for Streaming Video Understandingcited by 5
- 2026FOLIO: Focused Semantic Memory for Streaming Video Understandingcited by 4
- 2026StreamArena: Toward Continuous, Interactive, and Long-Horizon Agentic Streaming Video Understandingcited by 3
- 2026StreamFlow: Dynamic Memory Flows for Streaming Video Understandingcited by 1
- 2026Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understandingcited by 1
- 2026Omni-Streaming Thinkingcited by 1
- 2026OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interactioncited by 1
- 2026MoTE: Mixture of Task Experts for Multi-Task Video Understandingcited by 0
10 more
- 2026DeltaS: Reading the Gated Linear Attention State for KV Cache Eviction in Streaming Videocited by 0
- 2026KeyRec: Bounded Visual Memory for Streaming and Long-Video Understandingcited by 0
- 2026LLMs are General Asynchronous Agentscited by 0
- 2026EGSD: Event-Grounded Self-Distillation for Streaming Video Understandingcited by 0
- 2026Watch-Think-Interact: Bootstrapping Long-Horizon Multi-Turn Streaming Video Reasoning with Reinforcement Learningcited by 0
- 2026APM-Bench: Benchmarking Cross-session Persistent Memory for Egocentric Streaming Video Assistantscited by 0
- 2026MemLife: Curating and Reasoning over Long-Term Egocentric Video Memoriescited by 0
- 2026Foresight: planning future perception in streaming VLMs without retrainingcited by 0
- 2026EgoProbe: Benchmarking Long-Range and Exact Memory in Streaming Egocentric Videoscited by 0
- 2026DeltaStreamer: Towards Human-Like Agentic Streaming Video Understandingcited by 0
- 2026DeltaStreamer: Towards Human-Like Agentic Streaming Video Understanding14 Oct 2026
- 2026EgoProbe: Benchmarking Long-Range and Exact Memory in Streaming Egocentric Videos12 Oct 2026
- 2026Foresight: planning future perception in streaming VLMs without retraining2 Oct 2026
- 2026OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interaction1 Oct 2026
- 2026MemLife: Curating and Reasoning over Long-Term Egocentric Video Memories30 Sep 2026
- 2026APM-Bench: Benchmarking Cross-session Persistent Memory for Egocentric Streaming Video Assistants29 Sep 2026
- 2026Watch-Think-Interact: Bootstrapping Long-Horizon Multi-Turn Streaming Video Reasoning with Reinforcement Learning29 Sep 2026
- 2026EGSD: Event-Grounded Self-Distillation for Streaming Video Understanding29 Sep 2026
- 2026LLMs are General Asynchronous Agents28 Sep 2026
- 2026KeyRec: Bounded Visual Memory for Streaming and Long-Video Understanding26 Sep 2026
- 2026DeltaS: Reading the Gated Linear Attention State for KV Cache Eviction in Streaming Video23 Sep 2026
- 2026Omni-Streaming Thinking14 Sep 2026
10 more
- 2026Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understanding3 Sep 2026
- 2026MoTE: Mixture of Task Experts for Multi-Task Video Understanding25 Aug 2026
- 2026StreamFlow: Dynamic Memory Flows for Streaming Video Understanding11 Aug 2026
- 2026StreamArena: Toward Continuous, Interactive, and Long-Horizon Agentic Streaming Video Understanding6 Aug 2026
- 2026ObjectStream: Latent Objects as Memory Anchors for Streaming Video Understanding30 Jul 2026
- 2026FOLIO: Focused Semantic Memory for Streaming Video Understanding14 Jul 2026
- 2026JoyAI-VL-Interaction: Real-Time Vision-Language Interaction Intelligence10 Jun 2026
- 2026LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligence25 May 2026
- 2026Generation Models Know Space: Unleashing Implicit 3D Priors for Scene Understanding19 Mar 2026
- 2026Towards Generalizable Robotic Manipulation in Dynamic Environments16 Mar 2026
- 2026LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligencecited by 39
- 2026StreamFlow: Dynamic Memory Flows for Streaming Video Understandingcited by 1
- 2026Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understandingcited by 1
- 2026KeyRec: Bounded Visual Memory for Streaming and Long-Video Understandingcited by 0
- 2026Foresight: planning future perception in streaming VLMs without retrainingcited by 0
- 2026Foresight: planning future perception in streaming VLMs without retraining2 Oct 2026
- 2026KeyRec: Bounded Visual Memory for Streaming and Long-Video Understanding26 Sep 2026
- 2026Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understanding3 Sep 2026
- 2026StreamFlow: Dynamic Memory Flows for Streaming Video Understanding11 Aug 2026
- 2026LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligence25 May 2026
- 2026JoyAI-VL-Interaction: Real-Time Vision-Language Interaction Intelligencecited by 14
- 2026ObjectStream: Latent Objects as Memory Anchors for Streaming Video Understandingcited by 5
- 2026FOLIO: Focused Semantic Memory for Streaming Video Understandingcited by 4
- 2026Omni-Streaming Thinkingcited by 1
- 2026OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interactioncited by 1
- 2026Watch-Think-Interact: Bootstrapping Long-Horizon Multi-Turn Streaming Video Reasoning with Reinforcement Learningcited by 0
- 2026APM-Bench: Benchmarking Cross-session Persistent Memory for Egocentric Streaming Video Assistantscited by 0
- 2026MemLife: Curating and Reasoning over Long-Term Egocentric Video Memoriescited by 0
- 2026EgoProbe: Benchmarking Long-Range and Exact Memory in Streaming Egocentric Videoscited by 0
- 2026EgoProbe: Benchmarking Long-Range and Exact Memory in Streaming Egocentric Videos12 Oct 2026
- 2026OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interaction1 Oct 2026
- 2026MemLife: Curating and Reasoning over Long-Term Egocentric Video Memories30 Sep 2026
- 2026APM-Bench: Benchmarking Cross-session Persistent Memory for Egocentric Streaming Video Assistants29 Sep 2026
- 2026Watch-Think-Interact: Bootstrapping Long-Horizon Multi-Turn Streaming Video Reasoning with Reinforcement Learning29 Sep 2026
- 2026Omni-Streaming Thinking14 Sep 2026
- 2026ObjectStream: Latent Objects as Memory Anchors for Streaming Video Understanding30 Jul 2026
- 2026FOLIO: Focused Semantic Memory for Streaming Video Understanding14 Jul 2026
- 2026JoyAI-VL-Interaction: Real-Time Vision-Language Interaction Intelligence10 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2026Thinking in Streaming Video
- 2026Think While Watching: Online Streaming Segment-Level Memory for Multi-Turn Video Reasoning in Multimodal Large Language Models
- 2026StreamReady: Learning What to Answer and When in Long Streaming Videos
- 2026Think-as-You-See: Streaming Chain-of-Thought Reasoning for Large Vision-Language Models
- 2026WAT: Online Video Understanding Needs Watching Before Thinking
- 2025StreamingCoT: A Dataset for Temporal Dynamics and Multimodal Chain-of-Thought Reasoning in Streaming VideoQA
- 2026Progressive Online Video Understanding with Evidence-Aligned Timing and Transparent Decisions
- 2026X-Stream: Exploring MLLMs as Multiplexers for Multi-Stream Understanding