# VGR: Visual Grounded Reasoning

arXiv: https://arxiv.org/abs/2506.11991 · HTML: https://mlsanity.com/paper/2506.11991/

## Key facts

- Cited by 73 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 33 of its 63 references that are in the map.
- First posted on arXiv 13 Jun 2025.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 30 of 33)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [Evaluating Object Hallucination in Large Vision-Language Models](https://mlsanity.com/paper/2305.10355/) (2023)
- [ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://mlsanity.com/paper/2203.10244/) (2022)
- [Are We on the Right Way for Evaluating Large Vision-Language Models?](https://mlsanity.com/paper/2403.20330/) (2024)
- [DeepSeek-VL: Towards Real-World Vision-Language Understanding](https://mlsanity.com/paper/2403.05525/) (2024)
- [Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models](https://mlsanity.com/paper/2503.06749/) (2025)
- [DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding](https://mlsanity.com/paper/2412.10302/) (2024)
- [VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model](https://mlsanity.com/paper/2504.07615/) (2025)
- [Visual-RFT: Visual Reinforcement Fine-Tuning](https://mlsanity.com/paper/2503.01785/) (2025)
- [LLaVA-CoT: Let Vision Language Models Reason Step-by-Step](https://mlsanity.com/paper/2411.10440/) (2024)
- [Seed1.5-VL Technical Report](https://mlsanity.com/paper/2505.07062/) (2025)
- [R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization](https://mlsanity.com/paper/2503.10615/) (2025)
- [LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Models](https://mlsanity.com/paper/2407.12772/) (2024)
- [VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning](https://mlsanity.com/paper/2504.08837/) (2025)
- [Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization](https://mlsanity.com/paper/2411.10442/) (2024)
- [MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiency](https://mlsanity.com/paper/2502.09621/) (2025)
- [ZoomEye: Enhancing Multimodal LLMs with Human-Like Zooming Capabilities through Tree-Based Image Exploration](https://mlsanity.com/paper/2411.16044/) (2024)
- [Adaptive Chain-of-Focus Reasoning via Dynamic Visual Search and Zooming for Efficient VLMs](https://mlsanity.com/paper/2505.15436/) (2025)
- [ReFocus: Visual Editing as a Chain of Thought for Structured Image Understanding](https://mlsanity.com/paper/2501.05452/) (2025)
- [Benchmarking and Improving Detail Image Caption](https://mlsanity.com/paper/2405.19092/) (2024)
- [CogCoM: A Visual Language Model with Chain-of-Manipulations Reasoning](https://mlsanity.com/paper/2402.04236/) (2024)
- [Chain-of-Spot: Interactive Reasoning Improves Large Vision-Language Models](https://mlsanity.com/paper/2403.12966/) (2024)
- [Ross3D: Reconstructive Visual Instruction Tuning with 3D-Awareness](https://mlsanity.com/paper/2504.01901/) (2025)
- [Reconstructive Visual Instruction Tuning](https://mlsanity.com/paper/2410.09575/) (2024)
- [Scalable Vision Language Model Training via High Quality Data Curation](https://mlsanity.com/paper/2501.05952/) (2025)

## Built on by (most cited first, 30 of 73)

- [Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers](https://mlsanity.com/paper/2506.23918/) (2025)
- [DeepEyesV2: Toward Agentic Multimodal Model](https://mlsanity.com/paper/2511.05271/) (2025)
- [Latent Visual Reasoning](https://mlsanity.com/paper/2509.24251/) (2025)
- [Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoning](https://mlsanity.com/paper/2508.04416/) (2025)
- [Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation](https://mlsanity.com/paper/2605.18740/) (2026)
- [Traceable Evidence Enhanced Visual Grounded Reasoning: Evaluation and Methodology](https://mlsanity.com/paper/2507.07999/) (2025)
- [Look-Back: Implicit Visual Re-focusing in MLLM Reasoning](https://mlsanity.com/paper/2507.03019/) (2025)
- [Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception](https://mlsanity.com/paper/2602.11858/) (2026)
- [Open-o3-Video: Grounded Video Reasoning with Explicit Spatio-Temporal Evidence](https://mlsanity.com/paper/2510.20579/) (2025)
- [Reinforced Visual Perception with Tools](https://mlsanity.com/paper/2509.01656/) (2025)
- [VisMem: Latent Vision Memory Unlocks Potential of Vision-Language Models](https://mlsanity.com/paper/2511.11007/) (2025)
- [Beyond Seeing: Evaluating Multimodal LLMs on Tool-Enabled Image Perception, Transformation, and Reasoning](https://mlsanity.com/paper/2510.12712/) (2025)
- [From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Models](https://mlsanity.com/paper/2509.25373/) (2025)
- [Video-STAR: Reinforcing Open-Vocabulary Action Recognition with Tools](https://mlsanity.com/paper/2510.08480/) (2025)
- [AdaTooler-V: Adaptive Tool-Use for Images and Videos](https://mlsanity.com/paper/2512.16918/) (2025)
- [Grasp Any Region: Towards Precise, Contextual Pixel Understanding for Multimodal LLMs](https://mlsanity.com/paper/2510.18876/) (2025)
- [Latent Implicit Visual Reasoning](https://mlsanity.com/paper/2512.21218/) (2025)
- [DenseWorld-1M: Towards Detailed Dense Grounded Caption in the Real World](https://mlsanity.com/paper/2506.24102/) (2025)
- [SAIL-VL2 Technical Report](https://mlsanity.com/paper/2509.14033/) (2025)
- [Scheduling Your LLM Reinforcement Learning with Reasoning Trees](https://mlsanity.com/paper/2510.24832/) (2025)
- [Thinking With Bounding Boxes: Enhancing Spatio-Temporal Video Grounding via Reinforcement Fine-Tuning](https://mlsanity.com/paper/2511.21375/) (2025)
- [SAMTok: Representing Any Mask with Two Words](https://mlsanity.com/paper/2601.16093/) (2026)
- [On Robustness and Chain-of-Thought Consistency of RL-Finetuned VLMs](https://mlsanity.com/paper/2602.12506/) (2026)
- [SAIL-Embedding Technical Report: Omni-modal Embedding Foundation Model](https://mlsanity.com/paper/2510.12709/) (2025)
- [Can Textual Reasoning Improve the Performance of MLLMs on Fine-grained Visual Classification?](https://mlsanity.com/paper/2601.06993/) (2026)
- [PatchCue: Enhancing Vision-Language Model Reasoning with Patch-Based Visual Cues](https://mlsanity.com/paper/2603.05869/) (2026)
- [VisReason: A Large-Scale Dataset for Visual Chain-of-Thought Reasoning](https://mlsanity.com/paper/2511.17731/) (2025)
- [AdaReasoner: Dynamic Tool Orchestration for Iterative Visual Reasoning](https://mlsanity.com/paper/2601.18631/) (2026)
- [TerraScope: Pixel-Grounded Visual Reasoning for Earth Observation](https://mlsanity.com/paper/2603.19039/) (2026)
- [Thinking with Deltas: Incentivizing Reinforcement Learning via Differential Visual Reasoning Policy](https://mlsanity.com/paper/2601.06801/) (2026)

## Abstract (arXiv)

In the field of multimodal chain-of-thought (CoT) reasoning, existing approaches predominantly rely on reasoning on pure language space, which inherently suffers from language bias and is largely confined to math or science domains. This narrow focus limits their ability to handle complex visual reasoning tasks that demand comprehensive understanding of image details. To address these limitations, this paper introduces VGR, a novel reasoning multimodal large language model (MLLM) with enhanced fine-grained visual perception capabilities. Unlike traditional MLLMs that answer the question or reasoning solely on the language space, our VGR first detects relevant regions that may help to solve problems, and then provides precise answers based on replayed image regions. To achieve this, we conduct a large-scale SFT dataset called VGR -SFT that contains reasoning data with mixed vision grounding and language deduction. The inference pipeline of VGR allows the model to choose bounding boxes for visual reference and a replay stage is introduced to integrates the corresponding regions into the reasoning process, enhancing multimodel comprehension. Experiments on the LLaVA-NeXT-7B baseline show that VGR achieves superior performance on multi-modal benchmarks requiring comprehensive image detail understanding. Compared to the baseline, VGR uses only 30% of the image token count while delivering scores of +4.1 on MMStar, +7.1 on AI2D, and a +12.9 improvement on ChartQA.