RefBench-PRO: Perceptual and Reasoning Oriented Benchmark for Referring Expression Comprehension
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Referring Expression Comprehension (REC) is a vision-language task that localizes a specific image region based on a textual description. Existing REC benchmarks primarily evaluate perceptual capabilities and lack interpretable scoring mechanisms, which cannot reveal the grounding capability of Multi-modal Large Language Model (MLLM) across different cognitive abilities. To address this limitation, we introduce RefBench-PRO, a comprehensive REC benchmark, which decomposes referring expressions into two core dimensions, i.e., perception and reasoning, and further subdivides them into six progressively challenging tasks, such as attribute, position, interaction, commonsense, relation and reject. We also develop a fully automated data-generation pipeline that produces diverse referring expressions across these six sub-dimensions. Furthermore, We propose Ref-R1, an RL-based learning scheme, which incorporates Dynamic IoU-based GRPO to improve localization accuracy under increasingly complex reasoning conditions, establishing a stronger baseline for REC. Extensive experiments demonstrate that our RefBench-PRO enables interpretable evaluation of MLLM on referring expression comprehension, presenting greater challenges in both perception and reasoning.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 42 of its 47 references that are in the map. First posted on arXiv 6 Dec 2025.
Builds on
42 of 47 references.
Closest ancestors · 8
Same direction, newest first
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency25 Aug
- 2025Ovis2.5 Technical Report15 Aug
- 2025Rex-Thinker: Grounded Object Referring via Chain-of-Thought Reasoning4 Jun
- 2025UniVG-R1: Reasoning Guided Universal Visual Grounding with Reinforcement Learning20 May
- 2025Perception-R1: Pioneering Perception Policy with Reinforcement Learning10 Apr
- 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model10 Apr
- 2025Visual-RFT: Visual Reinforcement Fine-Tuning3 Mar
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
Other references · 34
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024GPT-4o System Cardcited by 4,588
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
- 2016Modeling Context in Referring Expressionscited by 1,063
- 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 904
22 more
- 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 708
- 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 584
- 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 572
- 2025LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Trainingcited by 244
- 2020Multi-task Collaborative Network for Joint Referring Expression Comprehension and Segmentationcited by 204
- 2024Ovis: Structural Embedding Alignment for Multimodal Large Language Modelcited by 169
- 2025MiMo: Unlocking the Reasoning Potential of Language Model -- From Pretraining to Posttrainingcited by 121
- 2024Groma: Localized Visual Tokenization for Grounding Multimodal Large Language Modelscited by 119
- 2025FG-CLIP: Fine-Grained Visual and Textual Alignmentcited by 81
- 2020Referring Expression Comprehension: A Survey of Methods and Datasetscited by 79
- 2024MM1.5: Methods, Analysis & Insights from Multimodal LLM Fine-tuningcited by 76
- 2023General Object Foundation Model for Images and Videos at Scalecited by 71
- 2025Eagle 2: Building Post-Training Data Strategies from Scratch for Frontier Vision-Language Modelscited by 70
- 2024Revisiting Referring Expression Comprehension Evaluation in the Era of Large Multimodal Modelscited by 53
- 2025Perceive Anything: Recognize, Explain, Caption, and Segment Anything in Images and Videoscited by 51
- 2023GREC: Generalized Referring Expression Comprehensioncited by 42
- 2024ChatRex: Taming Multimodal LLM for Joint Perception and Understandingcited by 30
- 2025Migician: Revealing the Magic of Free-Form Multi-Image Grounding in Multimodal Large Language Modelscited by 22
- 2025Referring to Any Personcited by 21
- 2024FineCops-Ref: A new Dataset and Task for Fine-Grained Compositional Referring Expression Comprehensioncited by 20
- 2025Patch-as-Decodable-Token: Towards Unified Multi-Modal Vision Tasks in MLLMscited by 20
- 2022Towards Unifying Reference Expression Generation and Comprehensioncited by 6
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Ref-Adv: Exploring MLLM Visual Reasoning in Referring Expression Tasks
- 2025KnowDR-REC: A Benchmark for Referring Expression Comprehension with Real-World Knowledge
- 2026Towards Open-World Referring Expression Comprehension: A Benchmark with Training-free Multi-task Consistency Checker
- 2025New Dataset and Methods for Fine-Grained Compositional Referring Expression Comprehension via Specialist-MLLM Collaboration
- 2026Chain-of-Caption: Training-free improvement of multimodal large language model on referring expression comprehension
- 2022One for All: One-stage Referring Expression Comprehension with Dynamic Reasoning
- 2022ReCLIP: A Strong Zero-Shot Baseline for Referring Expression Comprehension
- 2025Vision-Language Models Are Not Pragmatically Competent in Referring Expression Generation