Thinking in Streaming Video
Not on the map. Why?
Abstract (arXiv)
Real-time understanding of continuous video streams is essential for interactive assistants and multimodal agents operating in dynamic environments. However, most existing video reasoning approaches follow a batch paradigm that defers reasoning until the full video context is observed, resulting in high latency and growing computational cost that are incompatible with streaming scenarios. In this paper, we introduce ThinkStream, a framework for streaming video reasoning based on a Watch--Think--Speak paradigm that enables models to incrementally update their understanding as new video observations arrive. At each step, the model performs a short reasoning update and decides whether sufficient evidence has accumulated to produce a response. To support long-horizon streaming, we propose Reasoning-Compressed Streaming Memory (RCSM), which treats intermediate reasoning traces as compact semantic memory that replaces outdated visual tokens while preserving essential context. We further train the model using a Streaming Reinforcement Learning with Verifiable Rewards scheme that aligns incremental reasoning and response timing with the requirements of streaming interaction. Experiments on multiple streaming video benchmarks show that ThinkStream significantly outperforms existing online video models while maintaining low latency and memory usage. Code, models and data will be released at https://github.com/johncaged/ThinkStream
Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 45 of its 47 references that are in the map. First posted on arXiv 13 Mar 2026.
Builds on
45 of 47 references.
References · 45
- 2025Qwen3 Technical Reportcited by 8,195
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024Qwen2.5 Technical Reportcited by 4,713
- 2024GPT-4o System Cardcited by 4,467
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
- 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
- 2023VILA: On Pre-training for Visual Language Modelscited by 686
- 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 644
- 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 550
33 more
- 2024Long Context Transfer from Language to Visioncited by 502
- 2025Video-R1: Reinforcing Video Reasoning in MLLMscited by 464
- 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understandingcited by 341
- 2025FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Servingcited by 248
- 2024VideoLLM-online: Online Video Large Language Model for Streaming Videocited by 216
- 2024Flex Attention: A Programming Model for Generating Optimized Attention Kernelscited by 180
- 2024Streaming Long Video Understanding with Large Language Modelscited by 164
- 2024StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understandingcited by 135
- 2025OVO-Bench: How Far is Your Video-LLMs from Real-World Online Video Understanding?cited by 134
- 2024Liger Kernel: Efficient Triton Kernels for LLM Trainingcited by 123
- 2025Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reactioncited by 122
- 2025LightThinker: Thinking Step-by-Step Compressioncited by 104
- 2025StreamingVLM: Real-Time Understanding for Infinite Video Streamscited by 89
- 2025Scaling RL to Long Videoscited by 83
- 2025Tarsier2: Advancing Large Vision-Language Models from Detailed Video Description to Comprehensive Video Understandingcited by 80
- 2025Flash-VStream: Efficient Real-Time Understanding for Long Video Streamscited by 78
- 2025StreamForest: Efficient Online Video Understanding with Persistent Event Memorycited by 71
- 2025Ego-R1: Chain-of-Tool-Thought for Ultra-Long Egocentric Video Reasoningcited by 67
- 2025LION-FS: Fast & Slow Video-Language Thinker as Online Video Assistantcited by 56
- 2025StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistantcited by 56
- 2025StreamMem: Query-Agnostic KV Cache Memory for Streaming Video Understandingcited by 53
- 2025StreamMind: Unlocking Full Frame Rate Streaming Video Dialogue through Event-Gated Cognitioncited by 44
- 2026HERMES: KV Cache as Hierarchical Memory for Efficient Streaming Video Understandingcited by 36
- 2025Video-Thinker: Sparking "Thinking with Videos" via Reinforcement Learningcited by 32
- 2025Video-RTS: Rethinking Reinforcement Learning and Test-Time Scaling for Efficient and Enhanced Video Reasoningcited by 29
- 2025Streaming Video Instruction Tuningcited by 25
- 2025StreamKV: Streaming Video Question-Answering with Segment-based KV Cache Retrieval and Compressioncited by 24
- 2025TSPO: Temporal Sampling Policy Optimization for Long-form Video Language Understandingcited by 23
- 2025StreamingThinker: Large Language Models Can Think While Readingcited by 15
- 2025Breaking the Encoder Barrier for Seamless Video-Language Understandingcited by 8
- 2025EgoSpeak: Learning When to Speak for Egocentric Conversational Agents in the Wildcited by 7
- 2025TempR1: Improving Temporal Understanding of MLLMs via Temporal-Aware Multi-Task Reinforcement Learningcited by 7
- 2025CogStream: Context-guided Streaming Video Question Answeringcited by 5
Built on by
Citing papers per year: 2026: 14.
14 papers.
Sort:
By area:
- 2026A Simple Baseline for Streaming Video Understandingcited by 29
- 2026Semantic-Aware Adaptive Visual Memory for Streaming Video Understandingcited by 8
- 2026What Should a Streaming Video Model Remember?cited by 7
- 2026Toward Native Multimodal Modeling: A Roadmapcited by 6
- 2026StreamPro: From Reactive Perception to Proactive Decision-Making in Streaming Videocited by 3
- 2026FOLIO: Focused Semantic Memory for Streaming Video Understandingcited by 3
- 2026StreamArena: Toward Continuous, Interactive, and Long-Horizon Agentic Streaming Video Understandingcited by 3
- 2026Don't Pause: Streaming Video-Language Synchrony for Online Video Understandingcited by 1
- 2026StreamFlow: Dynamic Memory Flows for Streaming Video Understandingcited by 0
- 2026Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understandingcited by 0
- 2026Live Assistant: Learning Whether, When, and Whom to Assist in Real-World Live Social Streamscited by 0
- 2026TRACE: Temporal Audit and Condition-aware Evaluation of Streaming Video Understandingcited by 0
2 more
- 2026OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interaction1 Oct 2026
- 2026TRACE: Temporal Audit and Condition-aware Evaluation of Streaming Video Understanding25 Sep 2026
- 2026Live Assistant: Learning Whether, When, and Whom to Assist in Real-World Live Social Streams23 Sep 2026
- 2026Agentic Video Understanding: A Survey21 Sep 2026
- 2026Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understanding3 Sep 2026
- 2026StreamFlow: Dynamic Memory Flows for Streaming Video Understanding11 Aug 2026
- 2026StreamArena: Toward Continuous, Interactive, and Long-Horizon Agentic Streaming Video Understanding6 Aug 2026
- 2026FOLIO: Focused Semantic Memory for Streaming Video Understanding14 Jul 2026
- 2026What Should a Streaming Video Model Remember?15 Jun 2026
- 2026Don't Pause: Streaming Video-Language Synchrony for Online Video Understanding5 Jun 2026
- 2026Toward Native Multimodal Modeling: A Roadmap25 May 2026
- 2026StreamPro: From Reactive Perception to Proactive Decision-Making in Streaming Video11 May 2026
2 more
- 2026Toward Native Multimodal Modeling: A Roadmapcited by 6
- 2026Toward Native Multimodal Modeling: A Roadmap25 May 2026
- 2026A Simple Baseline for Streaming Video Understandingcited by 29
- 2026Semantic-Aware Adaptive Visual Memory for Streaming Video Understandingcited by 8
- 2026What Should a Streaming Video Model Remember?cited by 7
- 2026StreamPro: From Reactive Perception to Proactive Decision-Making in Streaming Videocited by 3
- 2026FOLIO: Focused Semantic Memory for Streaming Video Understandingcited by 3
- 2026Don't Pause: Streaming Video-Language Synchrony for Online Video Understandingcited by 1
- 2026Live Assistant: Learning Whether, When, and Whom to Assist in Real-World Live Social Streamscited by 0
- 2026TRACE: Temporal Audit and Condition-aware Evaluation of Streaming Video Understandingcited by 0
- 2026OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interactioncited by 0
- 2026OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interaction1 Oct 2026
- 2026TRACE: Temporal Audit and Condition-aware Evaluation of Streaming Video Understanding25 Sep 2026
- 2026Live Assistant: Learning Whether, When, and Whom to Assist in Real-World Live Social Streams23 Sep 2026
- 2026FOLIO: Focused Semantic Memory for Streaming Video Understanding14 Jul 2026
- 2026What Should a Streaming Video Model Remember?15 Jun 2026
- 2026Don't Pause: Streaming Video-Language Synchrony for Online Video Understanding5 Jun 2026
- 2026StreamPro: From Reactive Perception to Proactive Decision-Making in Streaming Video11 May 2026
- 2026Semantic-Aware Adaptive Visual Memory for Streaming Video Understanding8 May 2026
- 2026A Simple Baseline for Streaming Video Understanding2 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2026Think While Watching: Online Streaming Segment-Level Memory for Multi-Turn Video Reasoning in Multimodal Large Language Models
- 2026Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously
- 2026Think-as-You-See: Streaming Chain-of-Thought Reasoning for Large Vision-Language Models
- 2026AdaSR: Adaptive Streaming Reasoning with Hierarchical Relative Policy Optimization
- 2026StreamOPD: A Post-Training Recipe with Spatio-Temporal Cue Gating for Streaming Video Understanding
- 2026StreamReady: Learning What to Answer and When in Long Streaming Videos
- 2025StreamAgent: Towards Anticipatory Agents for Streaming Video Understanding
- 2026OASIS: On-Demand Hierarchical Event Memory for Streaming Video Reasoning