# When Thinking Drifts: Evidential Grounding for Robust Video Reasoning

arXiv: https://arxiv.org/abs/2510.06077 · HTML: https://mlsanity.com/paper/2510.06077/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 65 of its 75 references that are in the map.
- First posted on arXiv 7 Oct 2025.

## Builds on (most cited first, 30 of 65)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding](https://mlsanity.com/paper/2306.02858/) (2023)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [Least-to-Most Prompting Enables Complex Reasoning in Large Language Models](https://mlsanity.com/paper/2205.10625/) (2022)
- [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://mlsanity.com/paper/2501.12599/) (2025)
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [MVBench: A Comprehensive Multi-modal Video Understanding Benchmark](https://mlsanity.com/paper/2311.17005/) (2023)
- [Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks](https://mlsanity.com/paper/2211.12588/) (2022)
- [Show Your Work: Scratchpads for Intermediate Computation with Language Models](https://mlsanity.com/paper/2112.00114/) (2021)
- [VideoChat: Chat-Centric Video Understanding](https://mlsanity.com/paper/2305.06355/) (2023)
- [VILA: On Pre-training for Visual Language Models](https://mlsanity.com/paper/2312.07533/) (2023)
- [Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces](https://mlsanity.com/paper/2412.14171/) (2024)
- [Long Context Transfer from Language to Vision](https://mlsanity.com/paper/2406.16852/) (2024)
- [Video-R1: Reinforcing Video Reasoning in MLLMs](https://mlsanity.com/paper/2503.21776/) (2025)
- [Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model](https://mlsanity.com/paper/2503.24290/) (2025)

## Built on by (most cited first, 16 of 16)

- [Mull-Tokens: Modality-Agnostic Latent Thinking](https://mlsanity.com/paper/2512.10941/) (2025)
- [Video-R2: Reinforcing Consistent and Grounded Reasoning in Multimodal Language Models](https://mlsanity.com/paper/2511.23478/) (2025)
- [Think-as-You-See: Streaming Chain-of-Thought Reasoning for Large Vision-Language Models](https://mlsanity.com/paper/2603.02872/) (2026)
- [Don't Let the Video Speak: Audio-Contrastive Preference Optimization for Audio-Visual Language Models](https://mlsanity.com/paper/2604.14129/) (2026)
- [VKnowU: Evaluating Visual Knowledge Understanding in Multimodal LLMs](https://mlsanity.com/paper/2511.20272/) (2025)
- [GraphThinker: Reinforcing Temporally Grounded Video Reasoning with Event Graph Thinking](https://mlsanity.com/paper/2602.17555/) (2026)
- [HopChain: Multi-Hop Data Synthesis for Generalizable Vision-Language Reasoning](https://mlsanity.com/paper/2603.17024/) (2026)
- [When Thinking Hurts: Mitigating Visual Forgetting in Video Reasoning via Frame Repetition](https://mlsanity.com/paper/2603.16256/) (2026)
- [Learning What to Attend First: Modality-Importance-Guided Reasoning for Reliable Multimodal Emotion Understanding](https://mlsanity.com/paper/2512.02699/) (2025)
- [SHOE: Semantic HOI Open-Vocabulary Evaluation Metric](https://mlsanity.com/paper/2604.01586/) (2026)
- [Aligning Thoughts with Answers: Probability Rewards to Tame Thinking Drift](https://mlsanity.com/paper/2609.39183/) (2026)
- [IVR-R1: Refining Trajectories through Iterative Visual-Grounded Reasoning in Reinforcement Learning](https://mlsanity.com/paper/2605.23997/) (2026)
- [Be Faithful When Response: Returning Fluent and Grounded Answers for Vision-Language Models Reinforcement Learning](https://mlsanity.com/paper/2606.29984/) (2026)
- [Counterfactual Sensitivity Is Not Repairability: Auditing Replay Probes for Video Evidence](https://mlsanity.com/paper/2608.15685/) (2026)
- [Agentic Video Understanding: A Survey](https://mlsanity.com/paper/2609.31713/) (2026)
- [Meta-Video: Efficient Standardized Evaluation for Video Understanding](https://mlsanity.com/paper/2610.18535/) (2026)

## Abstract (arXiv)

Video reasoning, the task of enabling machines to infer from dynamic visual content through multi-step logic, is crucial for advanced AI. While the Chain-of-Thought (CoT) mechanism has enhanced reasoning in text-based tasks, its application to video understanding remains underexplored. This paper presents a systematic analysis revealing that CoT often degrades performance in video reasoning, generating verbose but misleading internal monologues, and leading to hallucinated visual details and overridden correct intuitions - a phenomenon we term "visual thinking drift". We explain this drift through a Bayesian lens, positing that CoT traces often diverge from actual visual evidence, instead amplifying internal biases or language priors, causing models to storytell rather than engage in grounded reasoning. To counteract this, we introduce Visual Evidence Reward (VER), a novel reinforcement learning framework that explicitly rewards the generation of reasoning traces that are verifiably grounded in visual evidence. Comprehensive evaluation across 10 diverse video understanding benchmarks demonstrates that our Video-VER consistently achieves top performance. Our work sheds light on the distinct challenges of video-centric reasoning and encourages the development of AI that robustly grounds its inferences in visual evidence - for large multimodal models that not only "think before answering", but also "see while thinking".