Scene Graph Thinking: Reinforcing Structured Visual Reasoning for Multimodal Large Language Models
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Multimodal Large Language Models (MLLMs) have demonstrated strong perception and reasoning capabilities. However, most existing models focus on isolated objects and neglect structured relationships for efficient target navigation, limiting their performance on visually intensive tasks. To address this challenge, we introduce Scene Graph Thinking (SaGe), a novel paradigm that enables fine-grained and structured visual reasoning through explicit scene-graph representations. Specifically, we first introduce an automated data engine that converts flat image-text corpora into structured scene graphs, where hierarchical entities constitute the nodes and diverse visual relations define the edges. Building upon this, we construct 120K high-quality training data by sampling reasoning traces from scene graphs. Then, two-stage graph-aligned post-training paradigms are introduced, where supervised fine-tuning internalizes MLLMs with structured reasoning, and subsequent reinforcement fine-tuning proposes node-as-proxy graph rewards to consolidate efficient graph exploration. With curated data and graph-aligned training, our approach achieves significant improvements across eight multimodal benchmarks, demonstrating strong effectiveness on fine-grained perception and reasoning tasks. Code is available at https://github.com/zwyang6/SaGe.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 29 of its 58 references that are in the map. First posted on arXiv 7 Jul 2026.
Builds on
29 of 58 references.
Closest ancestors · 12
Same direction, newest first
- 2025Latent Visual Reasoning29 Sep
- 2025Thyme: Think Beyond Images15 Aug
- 2025Zebra-CoT: A Dataset for Interleaved Vision Language Reasoning22 Jul
- 2025Look-Back: Implicit Visual Re-focusing in MLLM Reasoning2 Jul
- 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers30 Jun
- 2025Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learning21 May
- 2025GRIT: Teaching MLLMs to Think with Images21 May
- 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning20 May
- 2025VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning10 Apr
- 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model10 Apr
- 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models9 Mar
- 2025Visual-RFT: Visual Reinforcement Fine-Tuning3 Mar
Other references · 17
- 2025Qwen3 Technical Reportcited by 8,195
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024GPT-4o System Cardcited by 4,467
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
- 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
- 2024OpenAI o1 System Cardcited by 1,220
- 2025MLLMs Know Where to Look: Training-free Perception of Small Visual Details with Multimodal LLMscited by 172
5 more
- 2024GME: Improving Universal Multimodal Retrieval by Multimodal LLMscited by 166
- 2025DeepEyesV2: Toward Agentic Multimodal Modelcited by 119
- 2023Caption Anything: Interactive Image Description with Diverse Multimodal Controlscited by 98
- 2025OmniCaptioner: One Captioner to Rule Them Allcited by 19
- 2025Grasp Any Region: Towards Precise, Contextual Pixel Understanding for Multimodal LLMscited by 18
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026GraphVerse: A Comprehensive Visual Graph Reasoning Benchmark for Multimodal Large Language Models
- 2026SceneAlign: Aligning Multimodal Reasoning to Scene Graphs in Complex Visual Scenes
- 2025SpatialThinker: Reinforcing Scene Graph-Grounded Spatial Reasoning via Dense Rewards
- 2025Seeing Beyond the Scene: Enhancing Vision-Language Models with Interactional Reasoning
- 2026LookWise: Knowing When and Where to Look for Fine-Grained Visual Reasoning in Multimodal Large Language Models
- 2025EmbodiedVSR: Dynamic Scene Graph-Guided Chain-of-Thought Reasoning for Visual Spatial Tasks
- 2025Visual Graph Arena: Evaluating Visual Conceptualization of Vision and Multimodal Large Language Models
- 2024Benchmarking and Improving Large Vision-Language Models for Fundamental Visual Graph Understanding and Reasoning