mlsanityData to 7 Oct 2026

Groundbench: Multi-Resolution Polygon Grounding Exposes the Geometry Gap in Vision-Language Models

arXiv 2609.27821 ↗First posted 18 Aug 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Bounding-box scores on RefCOCO-family grounding leave little room to distinguish frontier vision-language systems, yet boxes discard object shape. We introduce GroundingBench, a matched benchmark that re-targets the same 1,500 image-expression-referent triples to exact-N polygons at five vertex budgets. A fixed-denominator harness separately audits filled-region intersection over union (IoU) and legal-polygon completion. The strongest tested configuration reaches 88.2 box IoU and 97.1 accuracy at IoU >= .5 ([email protected]), versus 57.7 and 69.2 for direct polygons; because these headline scores use different references, we also compare direct polygons with predicted boxes rasterised against the same contour target, obtaining 57.7 versus 57.3 when pooled. Performance is non-monotone in N and collapses at the densest budget, where legality failures compound residual geometric error. Qwen's thinking-setting contrast is the largest tested input-preserving configuration difference; under frozen templates, false spatial cues are more damaging than false colour cues, and target preference can remain high while contour tracing is poor. Alternate masks and a continuous-area scorer preserve the principal ordering. GroundingBench therefore measures an operational output-geometry gap spanning localisation, boundary construction, serialisation, and topology, rather than latent boundary perception alone.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 38 of its 43 references that are in the map. First posted on arXiv 18 Aug 2026.

Builds on

38 of 43 references.

References · 38

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2023Segment Anythingcited by 8,158
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2025Qwen2.5-VL Technical Reportcited by 5,418
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  7. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  8. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  9. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  10. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  11. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,075
  12. 2016Modeling Context in Referring Expressionscited by 1,054
26 more
  1. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 897
  2. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 891
  3. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 811
  4. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 770
  5. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
  6. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 529
  7. 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 499
  8. 2023Ferret: Refer and Ground Anything Anywhere at Any Granularitycited by 498
  9. 2023GLaMM: Pixel Grounding Large Multimodal Modelcited by 447
  10. 2021Pix2seq: A Language Modeling Framework for Object Detectioncited by 345
  11. 2020Are we done with ImageNet?cited by 329
  12. 2022Visual Spatial Reasoningcited by 288
  13. 2023GRES: Generalized Referring Expression Segmentationcited by 252
  14. 2023PolyFormer: Referring Image Segmentation as Sequential Polygon Generationcited by 147
  15. 2022SeqTR: A Simple yet Universal Network for Visual Groundingcited by 138
  16. 2018Efficient Interactive Annotation of Segmentation Datasets with Polygon-RNN++cited by 119
  17. 2020PhraseCut: Language-based Image Segmentation in the Wildcited by 116
  18. 2020Deep Snake for Real-Time Instance Segmentationcited by 111
  19. 2024GroundingGPT:Language Enhanced Multi-modal Grounding Modelcited by 101
  20. 2019Fast Interactive Object Annotation with Curve-GCNcited by 95
  21. 2017Annotating Object Instances with a Polygon-RNNcited by 88
  22. 2023NExT-Chat: An LMM for Chat, Detection and Segmentationcited by 79
  23. 2023Parallel Vertex Diffusion for Unified Visual Groundingcited by 28
  24. 2026GenSeg-R1: RL-Driven Vision-Language Grounding for Fine-Grained Referring Segmentationcited by 5
  25. 2026Qwen3-VL-Seg: Unlocking Open-World Referring Segmentation with Vision-Language Groundingcited by 5
  26. 2025RGBT-GroundBench: Visual Grounding Beyond RGB in Complex Real-World Scenarioscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Toward Comprehensive 3D Grounding: Orientation Grounding through Vision-Language Models
  2. 2026Grounded 3D-Aware Spatial Vision-Language Modeling
  3. 2025From Objects to Anywhere: A Holistic Benchmark for Multi-level Visual Grounding in 3D Scenes
  4. 2026UniGround: Universal 3D Visual Grounding via Training-Free Scene Parsing
  5. 2025OpenGround: Planning-based Online Perception for Open-World 3D Visual Grounding
  6. 2025DSM: Constructing a Diverse Semantic Map for 3D Visual Grounding
  7. 2026CoEvolve: Construct-to-Edit Visual Grounding with Bidirectional State Refinement
  8. 2026Spatial Reasoning is Not a Free Lunch: A Controlled Study on LLaVA