GraphThinker: Reinforcing Temporally Grounded Video Reasoning with Event Graph Thinking
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Video reasoning requires a fine-grained understanding of the temporal dependencies and event-level relations between objects and events in videos. Current Multimodal Large Language Models (MLLMs) are prone to severe temporal hallucinations in video reasoning. An underlying cause of these hallucinations is weak visual-temporal grounding and the lack of explicit structure for modelling event relations. Models often rely on auxiliary text, such as dense captions, rather than explicitly anchoring their reasoning in actual visual evidence. However, these textual representations are inherently unstructured and fail to provide explicit causal constraints needed to guide the model's reasoning. In this work, we propose GraphThinker, a reinforcement finetuning method that constructs a structured event representation of a video and enforces visual grounding to jointly reduce reasoning hallucinations. Specifically, we employ an MLLM to construct an Event-based Video Scene Graph (EVSG) that captures both intra- and inter-event relations, guiding a structured video reasoning process. Moreover, we address the weak grounding issue by introducing a novel visual attention reward during reinforcement finetuning that encourages the model to actively attend to reliable visual cues. On the RexTime dataset, GraphThinker achieves an over 4% improvement in IoU=0.3 for moment localisation. On the VidHalluc dataset, GraphThinker achieves a 9.8% improvement in reducing temporal sequence hallucination and a 7.6% gain in Binary QA in reducing action hallucination, compared to the state-of-the-art methods.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 67 of its 78 references that are in the map. First posted on arXiv 19 Feb 2026.
Builds on
67 of 78 references.
Closest ancestors · 19
Same direction, newest first
- 2025VTimeCoT: Thinking by Drawing for Video Temporal Grounding and Reasoning16 Oct
- 2025Look Again, Think Slowly: Enhancing Visual Reflection in Vision-Language Models15 Sep
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency25 Aug
- 2025Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoning6 Aug
- 2025Towards Video Thinking Test: A Holistic Benchmark for Advanced Video Reasoning and Understanding20 Jul
- 2025Scaling RL to Long Videos10 Jul
- 2025Video-CoT: A Comprehensive Dataset for Spatiotemporal Understanding of Videos Based on Chain-of-Thought10 Jun
- 2025Reinforcement Learning Tuning for VideoLLMs: Reward Design and Data Efficiency2 Jun
- 2025Chain-of-Frames: Advancing Video Understanding in Multimodal LLMs via Frame-Aware Reasoning31 May
- 2025Visionary-R1: Mitigating Shortcuts in Visual Reasoning with Reinforcement Learning20 May
- 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model10 Apr
- 2025VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuning9 Apr
7 more
- 2025Video-R1: Reinforcing Video Reasoning in MLLMs27 Mar
- 2025V-STaR: Benchmarking Video-LLMs on Video Spatio-Temporal Reasoning14 Mar
- 2024STEP: Enhancing Video-LLMs' Compositional Reasoning by Spatio-Temporal Graph-guided Self-Training29 Nov
- 2024VideoEspresso: A Large-Scale Chain-of-Thought Dataset for Fine-Grained Video Reasoning via Core Frame Selection22 Nov
- 2024Temporal Reasoning Transfer from Text to Video8 Oct
- 2024ReXTime: A Benchmark Suite for Reasoning-Across-Time in Videos27 Jun
- 2024Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognition7 May
Other references · 48
- 2023GPT-4 Technical Reportcited by 18,132
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
- 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
- 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
- 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
- 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 749
- 2023VILA: On Pre-training for Visual Language Modelscited by 686
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 580
- 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 550
- 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 502
36 more
- 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understandingcited by 399
- 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 386
- 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 368
- 2024PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioningcited by 320
- 2023VTimeLLM: Empower LLM to Grasp Video Momentscited by 253
- 2019Action Genome: Actions as Composition of Spatio-temporal Scene Graphscited by 249
- 2023UniVTG: Towards Unified Video-Language Temporal Groundingcited by 189
- 2024MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understandingcited by 180
- 2024Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Makingcited by 142
- 2025Time-R1: Post-Training Large Vision Language Model for Temporal Video Groundingcited by 138
- 2023Can I Trust Your Answer? Visually Grounded Video Question Answeringcited by 133
- 2024LITA: Language Instructed Temporal-Localization Assistantcited by 124
- 2021Spatial-Temporal Transformer for Dynamic Scene Graph Generationcited by 95
- 2024TRACE: Temporal Grounding Video LLM via Causal Event Modelingcited by 87
- 2025More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Modelscited by 84
- 2024Reka Core, Flash, and Edge: A Series of Powerful Multimodal Language Modelscited by 67
- 2024VidHalluc: Evaluating Temporal Hallucinations in Multimodal Large Language Models for Video Understandingcited by 65
- 2025Embodied AI Agents: Modeling the Worldcited by 57
- 2022Towards Open-vocabulary Scene Graph Generation with Prompt-based Finetuningcited by 54
- 2023Panoptic Video Scene Graph Generationcited by 46
- 2025SophiaVL-R1: Reinforcing MLLMs Reasoning with Thinking Rewardcited by 45
- 2024From Pixels to Graphs: Open-Vocabulary Scene Graph Generation with Vision-Language Modelscited by 43
- 2023Unbiased Scene Graph Generation in Videoscited by 38
- 2025PerturboLLaVA: Reducing Multimodal Hallucinations with Perturbative Visual Trainingcited by 37
- 2024Do You Remember? Dense Video Captioning with Cross-Modal Memory Retrievalcited by 36
- 2023Correlation-Guided Query-Dependency Calibration for Video Temporal Groundingcited by 31
- 2023HIG: Hierarchical Interlacement Graph Approach to Scene Graph Generation in Video Understandingcited by 21
- 2025Building a Mind Palace: Structuring Environment-Grounded Semantic Graphs for Effective Long Video Analysis with LLMscited by 19
- 2024HyperGLM: HyperGraph for Video Scene Graph Generation and Anticipationcited by 17
- 2025When Thinking Drifts: Evidential Grounding for Robust Video Reasoningcited by 15
- 2024SportsHHI: A Dataset for Human-Human Interaction Detection in Sports Videoscited by 14
- 2025Dr.V: A Hierarchical Perception-Temporal-Cognition Framework to Diagnose Video Hallucination by Fine-grained Spatial-Temporal Groundingcited by 13
- 2024Question-Answering Dense Video Eventscited by 11
- 2025MUSEG: Reinforcing Video Temporal Understanding via Timestamp-Aware Multi-Segment Groundingcited by 11
- 2025ZoomV: Temporal Zoom-in for Efficient Long Video Understandingcited by 10
- 2025TOGA: Temporally Grounded Open-Ended Video QA with Weak Supervisioncited by 8
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligencecited by 31
- 2026Plan-and-Verify Video Reward Reasoning with Spatio-Temporal Scene Graph Groundingcited by 0
- 2026EventCoT: Event-centric Video Chain-of-thought for Reasoning Temporal Localizationcited by 0
- 2026Video-FLAIR: Not Whether to Reason, But Howcited by 0
- 2026Locate Anything in Videos: Rethinking Efficient Generative Spatio-Temporal Video Groundingcited by 0
- 2026Locate Anything in Videos: Rethinking Efficient Generative Spatio-Temporal Video Grounding28 Aug 2026
- 2026Video-FLAIR: Not Whether to Reason, But How27 Aug 2026
- 2026EventCoT: Event-centric Video Chain-of-thought for Reasoning Temporal Localization6 Jul 2026
- 2026Plan-and-Verify Video Reward Reasoning with Spatio-Temporal Scene Graph Grounding10 Jun 2026
- 2026LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligence25 May 2026
Similar, not cited
Close in wording, no citation either way
- 2025Video Finetuning Improves Reasoning Between Frames
- 2023Look, Remember and Reason: Grounded reasoning in videos with language models
- 2025Video-R2: Reinforcing Consistent and Grounded Reasoning in Multimodal Language Models
- 2026Moment-Video: Diagnosing Temporal Fidelity of Video MLLMs on Momentary Visual Events
- 2026TimeThink: Reasoning with Time for Video LLMs
- 2025TAR: Temporal Anchor-Constrained Reasoning for Video Temporal Grounding
- 2025VCRBench: Exploring Long-form Causal Reasoning Capabilities of Large Video Language Models
- 2025FrameThinker: Learning to Think with Long Videos via Multi-Turn Frame Spotlighting