# Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously

arXiv: https://arxiv.org/abs/2603.12262 · HTML: https://mlsanity.com/paper/2603.12262/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 58 of its 70 references that are in the map.
- First posted on arXiv 12 Mar 2026.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 30 of 58)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [PaLM-E: An Embodied Multimodal Language Model](https://mlsanity.com/paper/2303.03378/) (2023)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [Understanding R1-Zero-Like Training: A Critical Perspective](https://mlsanity.com/paper/2503.20783/) (2025)
- [MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies](https://mlsanity.com/paper/2404.06395/) (2024)
- [VILA: On Pre-training for Visual Language Models](https://mlsanity.com/paper/2312.07533/) (2023)
- [LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding](https://mlsanity.com/paper/2407.15754/) (2024)
- [PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel](https://mlsanity.com/paper/2304.11277/) (2023)
- [LLaVA-Video: Video Instruction Tuning With Synthetic Data](https://mlsanity.com/paper/2410.02713/) (2024)
- [MovieChat: From Dense Token to Sparse Memory for Long Video Understanding](https://mlsanity.com/paper/2307.16449/) (2023)
- [Long Context Transfer from Language to Vision](https://mlsanity.com/paper/2406.16852/) (2024)
- [Video-R1: Reinforcing Video Reasoning in MLLMs](https://mlsanity.com/paper/2503.21776/) (2025)
- [Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Models](https://mlsanity.com/paper/2311.06607/) (2023)
- [LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understanding](https://mlsanity.com/paper/2410.17434/) (2024)
- [LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Models](https://mlsanity.com/paper/2407.12772/) (2024)
- [MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent](https://mlsanity.com/paper/2507.02259/) (2025)
- [VideoLLM-online: Online Video Large Language Model for Streaming Video](https://mlsanity.com/paper/2406.11816/) (2024)
- [Streaming Long Video Understanding with Large Language Models](https://mlsanity.com/paper/2405.16009/) (2024)
- [StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understanding](https://mlsanity.com/paper/2411.03628/) (2024)
- [OVO-Bench: How Far is Your Video-LLMs from Real-World Online Video Understanding?](https://mlsanity.com/paper/2501.05510/) (2025)
- [Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reaction](https://mlsanity.com/paper/2501.03218/) (2025)
- [Video-Holmes: Can MLLM Think Like Holmes for Complex Video Reasoning?](https://mlsanity.com/paper/2505.21374/) (2025)

## Built on by (most cited first, 22 of 22)

- [LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligence](https://mlsanity.com/paper/2605.25979/) (2026)
- [Towards Generalizable Robotic Manipulation in Dynamic Environments](https://mlsanity.com/paper/2603.15620/) (2026)
- [JoyAI-VL-Interaction: Real-Time Vision-Language Interaction Intelligence](https://mlsanity.com/paper/2606.14777/) (2026)
- [Generation Models Know Space: Unleashing Implicit 3D Priors for Scene Understanding](https://mlsanity.com/paper/2603.19235/) (2026)
- [ObjectStream: Latent Objects as Memory Anchors for Streaming Video Understanding](https://mlsanity.com/paper/2607.28312/) (2026)
- [FOLIO: Focused Semantic Memory for Streaming Video Understanding](https://mlsanity.com/paper/2607.13298/) (2026)
- [StreamArena: Toward Continuous, Interactive, and Long-Horizon Agentic Streaming Video Understanding](https://mlsanity.com/paper/2608.05703/) (2026)
- [StreamFlow: Dynamic Memory Flows for Streaming Video Understanding](https://mlsanity.com/paper/2608.10949/) (2026)
- [Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understanding](https://mlsanity.com/paper/2609.04131/) (2026)
- [Omni-Streaming Thinking](https://mlsanity.com/paper/2609.15128/) (2026)
- [OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interaction](https://mlsanity.com/paper/2610.01762/) (2026)
- [MoTE: Mixture of Task Experts for Multi-Task Video Understanding](https://mlsanity.com/paper/2608.24763/) (2026)
- [DeltaS: Reading the Gated Linear Attention State for KV Cache Eviction in Streaming Video](https://mlsanity.com/paper/2609.27470/) (2026)
- [KeyRec: Bounded Visual Memory for Streaming and Long-Video Understanding](https://mlsanity.com/paper/2609.32182/) (2026)
- [LLMs are General Asynchronous Agents](https://mlsanity.com/paper/2609.35427/) (2026)
- [EGSD: Event-Grounded Self-Distillation for Streaming Video Understanding](https://mlsanity.com/paper/2609.36803/) (2026)
- [Watch-Think-Interact: Bootstrapping Long-Horizon Multi-Turn Streaming Video Reasoning with Reinforcement Learning](https://mlsanity.com/paper/2609.37035/) (2026)
- [APM-Bench: Benchmarking Cross-session Persistent Memory for Egocentric Streaming Video Assistants](https://mlsanity.com/paper/2609.37559/) (2026)
- [MemLife: Curating and Reasoning over Long-Term Egocentric Video Memories](https://mlsanity.com/paper/2609.40195/) (2026)
- [Foresight: planning future perception in streaming VLMs without retraining](https://mlsanity.com/paper/2610.03123/) (2026)
- [EgoProbe: Benchmarking Long-Range and Exact Memory in Streaming Egocentric Videos](https://mlsanity.com/paper/2610.17128/) (2026)
- [DeltaStreamer: Towards Human-Like Agentic Streaming Video Understanding](https://mlsanity.com/paper/2610.19849/) (2026)

## Abstract (arXiv)

Online Video Large Language Models (VideoLLMs) play a critical role in supporting responsive, real-time interaction. Existing methods focus on streaming perception, lacking a synchronized logical reasoning stream. However, directly applying test-time scaling methods incurs unacceptable response latency. To address this trade-off, we propose Video Streaming Thinking (VST), a novel paradigm for streaming video understanding. It supports a thinking while watching mechanism, which activates reasoning over incoming video clips during streaming. This design improves timely comprehension and coherent cognition while preserving real-time responsiveness by amortizing LLM reasoning latency over video playback. Furthermore, we introduce a comprehensive post-training pipeline that integrates VST-SFT, which structurally adapts the offline VideoLLM to causal streaming reasoning, and VST-RL, which provides end-to-end improvement through self-exploration in a multi-turn video interaction environment. Additionally, we devise an automated training-data synthesis pipeline that uses video knowledge graphs to generate high-quality streaming QA pairs, with an entity-relation grounded streaming Chain-of-Thought to enforce multi-evidence reasoning and sustained attention to the video stream. Extensive evaluations show that VST-7B performs strongly on online benchmarks, e.g. 79.5% on StreamingBench and 59.3% on OVO-Bench. Meanwhile, VST remains competitive on offline long-form or reasoning benchmarks. Compared with Video-R1, VST responds 15.7 times faster and achieves +5.4% improvement on VideoHolmes, demonstrating higher efficiency and strong generalization across diverse video understanding tasks. Code, data, and models will be released at https://github.com/1ranGuan/VST.