# Where Grounding Accuracy Lives on the IoU Curve: Label-Free Inference-Time Boundary Refinement

arXiv: https://arxiv.org/abs/2608.19553 · HTML: https://mlsanity.com/paper/2608.19553/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 21 references that are in the map.
- First posted on arXiv 20 Aug 2026.

## Builds on (most cited first, 16 of 16)

- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models](https://mlsanity.com/paper/1505.04870/) (2015)
- [Grounded Language-Image Pre-training](https://mlsanity.com/paper/2112.03857/) (2021)
- [Modeling Context in Referring Expressions](https://mlsanity.com/paper/1608.00272/) (2016)
- [Generation and Comprehension of Unambiguous Object Descriptions](https://mlsanity.com/paper/1511.02283/) (2015)
- [MDETR -- Modulated Detection for End-to-End Multi-Modal Understanding](https://mlsanity.com/paper/2104.12763/) (2021)
- [GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning](https://mlsanity.com/paper/2507.01006/) (2025)
- [Scaling Open-Vocabulary Object Detection](https://mlsanity.com/paper/2306.09683/) (2023)
- [Revisiting Referring Expression Comprehension Evaluation in the Era of Large Multimodal Models](https://mlsanity.com/paper/2406.16866/) (2024)
- [Multi-task Visual Grounding with Coarse-to-Fine Consistency Constraints](https://mlsanity.com/paper/2501.06710/) (2025)
- [Entropy-Gradient Grounding: Training-Free Evidence Retrieval in Vision-Language Models](https://mlsanity.com/paper/2604.08456/) (2026)
- [Chain-of-Caption: Training-free improvement of multimodal large language model on referring expression comprehension](https://mlsanity.com/paper/2602.08211/) (2026)
- [Test-time Scaling over Perception: Resolving the Grounding Paradox in Thinking with Images](https://mlsanity.com/paper/2604.11025/) (2026)
- [Iterative Visual Thinking and the Self-Correction Mirage in VLM Grounding](https://mlsanity.com/paper/2606.13156/) (2026)

## Abstract (arXiv)

Vision--language models can identify the correct referent while returning an imprecise bounding box. We study whether a frozen direct-answer model can use its own prediction to allocate one additional localized observation without accessing target annotations at inference. Label-free precision refinement (LFPR) routes predicted-small regions to a higher-resolution pass, re-grounds the expression inside a context crop, admits a candidate only under fixed geometric guards, and returns a fixed coordinate-wise midpoint. We report results across three evidence tiers. On 31,921 retrospective Ref-L4 expressions, LFPR raises mAcc0.5:0.95 from 72.947% to 76.013% (Acc@0.5 88.531%→89.725%, Acc@0.9 55.788%→61.142%). A frozen transfer to 30,969 RefCOCO/RefCOCO+/RefCOCOg expressions improves every dataset at Acc@0.5, mAcc, and mean IoU (pooled mAcc +0.645, Acc@0.5 +0.817), while Acc@0.9 is unchanged overall: routing alone gains +1.162 points there, but crop, guards, and fusion give back -1.192, offsetting rather than showing no strict-IoU effect. A prospective, image-disjoint Flickr30K Entities evaluation improves every endpoint (mAcc +0.973, Acc@0.9 +1.022), more strongly under a single-box variant (mAcc +2.575, Acc@0.9 +3.689). The same operator applied to two released grounding specialists improves every endpoint (Acc@0.9 +1.569/+6.716 for EGM-4B/8B) at roughly twice the latency, composing with specialist training rather than replacing it. A genuine unguarded control (guard removed from the same candidates) underperforms the incumbent on every metric, showing the guard is load-bearing. Together, these results show that referent selection and boundary precision are partially separable, with different components moving opposing regions of the IoU curve -- behavior a single threshold cannot reveal.