Do Thought Streams Matter? Evaluating Reasoning in Gemini Vision-Language Models for Video Scene Understanding
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
We benchmark how internal reasoning traces, which we call thought streams, affect video scene understanding in vision-language models. Using four configurations of Google's Gemini 2.5 Flash and Flash Lite across scenes extracted from 100 hours of video, we ask three questions: does more thinking lead to better outputs, where do the gains stop, and what do these models actually think about? We introduce three evaluation metrics. Contentfulness measures how much of the thought stream is useful scene content versus meta-commentary. Thought-Final Coverage measures how faithfully the thought stream translates into the final output. Dominant Entity Analysis identifies which subjects, actions, and settings the model focuses on. GPT-5 serves as an independent judge. We find that quality gains from additional thinking plateau quickly, with most improvement occurring in the first few hundred tokens. Flash Lite offers the best balance between quality and token usage. Tight reasoning budgets cause the model to add content in the final output that it never reasoned about, a form of compression-step hallucination. Despite being different model tiers, Flash and Flash Lite produce similar thought streams, though they differ in style: Flash discusses its reasoning process, while Lite focuses on describing the scene.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 9 of its 15 references that are in the map. First posted on arXiv 13 Apr 2026.
Builds on
9 of 15 references.
References · 9
- 2023GPT-4 Technical Reportcited by 18,396
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Visual Thoughts: A Unified Perspective of Understanding Multimodal Chain-of-Thought
- 2023Let's Think Frame by Frame with VIP: A Video Infilling and Prediction Dataset for Evaluating Video Chain-of-Thought
- 2025BLINK-Twice: You see, but do you observe? A Reasoning Benchmark on Visual Perception
- 2026Reasoning Dynamics and the Limits of Monitoring Modality Reliance in Vision-Language Models
- 2026Act2See: Emergent Active Visual Perception for Video Reasoning
- 2026CinematicVQA: Benchmarking Film-Grammar Reasoning in Large Vision-Language Models
- 2026Watch, Remember, Reason: Human-View Video Understanding with MLLMs
- 2024VISCO: Benchmarking Fine-Grained Critique and Correction Towards Self-Improvement in Visual Reasoning