DiG: Differential Grounding for Enhancing Fine-Grained Perception in Multimodal Large Language Model
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Multimodal Large Language Models have achieved impressive performance on a variety of vision-language tasks, yet their fine-grained visual perception and precise spatial reasoning remain limited. In this work, we introduce DiG (Differential Grounding), a novel proxy task framework where MLLMs learn fine-grained perception by identifying and localizing all differences between similar image pairs without prior knowledge of their number. To support scalable training, we develop an automated 3D rendering-based data generation pipeline that produces high-quality paired images with fully controllable discrepancies. To address the sparsity of difference signals, we further employ curriculum learning that progressively increases complexity from single to multiple differences, enabling stable optimization. Extensive experiments demonstrate that DiG significantly improves model performance across a variety of visual perception benchmarks and that the learned fine-grained perception skills transfer effectively to standard downstream tasks, including RefCOCO, RefCOCO+, RefCOCOg, and general multimodal perception benchmarks. Our results highlight differential grounding as a scalable and robust approach for advancing fine-grained visual reasoning in MLLMs.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 47 of its 54 references that are in the map. First posted on arXiv 14 Dec 2025.
Builds on
47 of 54 references.
Closest ancestors · 9
Same direction, newest first
- 2025Agentic Jigsaw Interaction Learning for Enhancing Visual Perception and Reasoning in Vision-Language Models1 Oct
- 2025Visual Jigsaw Post-Training Improves MLLMs29 Sep
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency25 Aug
- 2025ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMs11 Jun
- 2025Perception-R1: Pioneering Perception Policy with Reinforcement Learning10 Apr
- 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models9 Mar
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
- 2023MMBench: Is Your Multi-modal Model an All-around Player?12 Jul
- 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering20 Sep
Other references · 38
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024GPT-4o System Cardcited by 4,467
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2023Qwen Technical Reportcited by 3,273
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
- 2019Towards VQA Models That Can Readcited by 1,696
26 more
- 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
- 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
- 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
- 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
- 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
- 2016Modeling Context in Referring Expressionscited by 1,054
- 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
- 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
- 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 897
- 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
- 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
- 2016A Diagram Is Worth A Dozen Imagescited by 793
- 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
- 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 572
- 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 459
- 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimizationcited by 296
- 2022Visual Spatial Reasoningcited by 288
- 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learningcited by 260
- 2024Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Modelscited by 151
- 2025Enigmata: Scaling Logical Reasoning in Large Language Models with Synthetic Verifiable Puzzlescited by 53
- 2025Reinforcement Pre-Trainingcited by 40
- 2024HRVDA: High-Resolution Visual Document Assistantcited by 37
- 2025Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with Jigsaw Puzzlescited by 36
- 2023Attention Where It Matters: Rethinking Visual Document Understanding with Selective Region Concentrationcited by 18
- 2025THOR: Tool-Integrated Hierarchical Optimization via RL for Mathematical Reasoningcited by 9
- 2025Input Domain Aware MoE: Decoupling Routing Decisions from Task Optimization in Mixture of Expertscited by 3
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perceptioncited by 47
- 2026Dynamic Token Compression for Efficient Video Understanding through Reinforcement Learningcited by 1
- 2026FORUM: Frozen Outputs Reconciled Using Model Agreement for Visual Groundingcited by 0
- 2026FORUM: Frozen Outputs Reconciled Using Model Agreement for Visual Grounding27 Sep 2026
- 2026Dynamic Token Compression for Efficient Video Understanding through Reinforcement Learning27 Mar 2026
- 2026Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception12 Feb 2026
Similar, not cited
Close in wording, no citation either way
- 2026PGT: Procedurally Generated Tasks for improving visual grounding in MLLMs
- 2026GeM-VG: Towards Generalized Multi-image Visual Grounding with Multimodal Large Language Models
- 2025Hierarchical Contextual Grounding LVLM: Enhancing Fine-Grained Visual-Language Understanding with Robust Grounding
- 2026Think with Structured Grounding: Perceptual Reinforcement Learning for Chart and Visual-Tabular Understanding
- 2026LOCUS: Local Visual Cue Search for Enhancing Fine-Grained Perception in Multimodal Large Language Models
- 2024MC-Bench: A Benchmark for Multi-Context Visual Grounding in the Era of MLLMs
- 2026Towards Understanding Multimodal Fine-Tuning: Spatial Features
- 2026Mixture-of-Thought-Tokens: Unifying Perception and Reasoning for Free-form Multimodal Grounding