Groundbench: Multi-Resolution Polygon Grounding Exposes the Geometry Gap in Vision-Language Models
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
Abstract (arXiv)
Bounding-box scores on RefCOCO-family grounding leave little room to distinguish frontier vision-language systems, yet boxes discard object shape. We introduce GroundingBench, a matched benchmark that re-targets the same 1,500 image-expression-referent triples to exact-N polygons at five vertex budgets. A fixed-denominator harness separately audits filled-region intersection over union (IoU) and legal-polygon completion. The strongest tested configuration reaches 88.2 box IoU and 97.1 accuracy at IoU >= .5 ([email protected]), versus 57.7 and 69.2 for direct polygons; because these headline scores use different references, we also compare direct polygons with predicted boxes rasterised against the same contour target, obtaining 57.7 versus 57.3 when pooled. Performance is non-monotone in N and collapses at the densest budget, where legality failures compound residual geometric error. Qwen's thinking-setting contrast is the largest tested input-preserving configuration difference; under frozen templates, false spatial cues are more damaging than false colour cues, and target preference can remain high while contour tracing is poor. Alternate masks and a continuous-area scorer preserve the principal ordering. GroundingBench therefore measures an operational output-geometry gap spanning localisation, boundary construction, serialisation, and topology, rather than latent boundary perception alone.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 38 of its 43 references that are in the map. First posted on arXiv 18 Aug 2026.
Builds on
38 of 43 references.
References · 38
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2023Segment Anythingcited by 8,158
- 2020Measuring Massive Multitask Language Understandingcited by 7,768
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
- 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
- 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
- 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
- 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,075
- 2016Modeling Context in Referring Expressionscited by 1,054
26 more
- 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 897
- 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 891
- 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 811
- 2023LISA: Reasoning Segmentation via Large Language Modelcited by 770
- 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
- 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 529
- 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 499
- 2023Ferret: Refer and Ground Anything Anywhere at Any Granularitycited by 498
- 2023GLaMM: Pixel Grounding Large Multimodal Modelcited by 447
- 2021Pix2seq: A Language Modeling Framework for Object Detectioncited by 345
- 2020Are we done with ImageNet?cited by 329
- 2022Visual Spatial Reasoningcited by 288
- 2023GRES: Generalized Referring Expression Segmentationcited by 252
- 2023PolyFormer: Referring Image Segmentation as Sequential Polygon Generationcited by 147
- 2022SeqTR: A Simple yet Universal Network for Visual Groundingcited by 138
- 2018Efficient Interactive Annotation of Segmentation Datasets with Polygon-RNN++cited by 119
- 2020PhraseCut: Language-based Image Segmentation in the Wildcited by 116
- 2020Deep Snake for Real-Time Instance Segmentationcited by 111
- 2024GroundingGPT:Language Enhanced Multi-modal Grounding Modelcited by 101
- 2019Fast Interactive Object Annotation with Curve-GCNcited by 95
- 2017Annotating Object Instances with a Polygon-RNNcited by 88
- 2023NExT-Chat: An LMM for Chat, Detection and Segmentationcited by 79
- 2023Parallel Vertex Diffusion for Unified Visual Groundingcited by 28
- 2026GenSeg-R1: RL-Driven Vision-Language Grounding for Fine-Grained Referring Segmentationcited by 5
- 2026Qwen3-VL-Seg: Unlocking Open-World Referring Segmentation with Vision-Language Groundingcited by 5
- 2025RGBT-GroundBench: Visual Grounding Beyond RGB in Complex Real-World Scenarioscited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Toward Comprehensive 3D Grounding: Orientation Grounding through Vision-Language Models
- 2026Grounded 3D-Aware Spatial Vision-Language Modeling
- 2025From Objects to Anywhere: A Holistic Benchmark for Multi-level Visual Grounding in 3D Scenes
- 2026UniGround: Universal 3D Visual Grounding via Training-Free Scene Parsing
- 2025OpenGround: Planning-based Online Perception for Open-World 3D Visual Grounding
- 2025DSM: Constructing a Diverse Semantic Map for 3D Visual Grounding
- 2026CoEvolve: Construct-to-Edit Visual Grounding with Bidirectional State Refinement
- 2026Spatial Reasoning is Not a Free Lunch: A Controlled Study on LLaVA