Reasoning-Guided Part-Level Visual Grounding via Reinforcement Learning
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Multimodal large language models (MLLMs) ground whole objects well from free-form language queries, but they struggle when the query names a part rather than the object. We trace this to a missing object-part hierarchy, since parts are localized in the same single step used for objects. We propose Object-Part Hierarchical Reflective Grounding (OP-HRG), a coarse-to-fine reasoning-guided grounding strategy that first localizes the parent object and then the part within it. A self-check then reflects on the result, with an extension to re-encode the predicted crop to inspect the region it is correcting. We introduce a part-aware GRPO framework to train our pipeline with stage-wise rewards. A 4B model trained this way outperforms 7B grounding LLMs and SAM3 across PascalPart, PartImageNet, and InstructPart, and transfers to reasoning segmentation.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 21 of its 61 references that are in the map. First posted on arXiv 16 Jul 2026.
Builds on
21 of 61 references.
Closest ancestors · 10
Same direction, newest first
- 2026DR2Seg: Decomposed Two-Stage Rollouts for Efficient Reasoning Segmentation in Multimodal Large Language Models15 Jan
- 2025Seg-R1: Segmentation Can Be Surprisingly Simple with Reinforcement Learning27 Jun
- 2025Ground-R1: Incentivizing Grounded Visual Reasoning via Reinforcement Learning26 May
- 2025SATORI-R1: Incentivizing Multimodal Reasoning through Explicit Visual Anchoring25 May
- 2025UniVG-R1: Reasoning Guided Universal Visual Grounding with Reinforcement Learning20 May
- 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model10 Apr
- 2025KARL: Knowledge-Aware Reasoning and Reinforcement Learning for Knowledge-Intensive Visual Grounding17 Mar
- 2025MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learning10 Mar
- 2025Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement9 Mar
- 2023LISA++: An Improved Baseline for Reasoning Segmentation with Large Language Model28 Dec
Other references · 11
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
- 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
- 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 891
- 2025Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videoscited by 154
- 2026GenSeg-R1: RL-Driven Vision-Language Grounding for Fine-Grained Referring Segmentationcited by 5
- 2026EGM: Efficient Visual Grounding Language Modelscited by 2
- 2026AffordanceGrasp-R1:Leveraging Reasoning-Based Affordance Segmentation with Reinforcement Learning for Robotic Graspingcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Connecting the Dots: Training-Free Visual Grounding via Agentic Reasoning
- 2024GROUNDHOG: Grounding Large Language Models to Holistic Segmentation
- 2026H-GRPO: Permutation-Invariant Reinforcement Learning for Grounded Visual Reasoning
- 2025No Labels, No Problem: Training Visual Reasoners with Multimodal Verifiers
- 2026Mixture-of-Thought-Tokens: Unifying Perception and Reasoning for Free-form Multimodal Grounding
- 2026GeM-VG: Towards Generalized Multi-image Visual Grounding with Multimodal Large Language Models
- 2023Language-Guided Diffusion Model for Visual Grounding
- 2026Thinking with Visual Grounding