GAVEL: Grounded Caption Error Verification and Localization
On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning
Abstract (arXiv)
Vision-language models (VLMs) often produce hallucinated or inconsistent outputs, where text and images are not properly aligned. Addressing this issue requires not only detecting misalignment but also explaining the discrepancy and localizing its visual evidence. We introduce GAVEL (Grounded Caption Error Verification and Localization), a task that jointly addresses verification, explanation, and localization for image-text pairs. To support systematic evaluation, we also present a corresponding dataset and benchmark. We further train a supervised baseline on the human-annotated training split to assess whether GAVEL provides learnable supervision for these abilities. Experiments show that even strong closed-source models struggle on GAVEL, while the supervised baseline yields consistent improvements across grounding and explanation metrics.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 9 of its 16 references that are in the map. First posted on arXiv 25 Jun 2026.
Builds on
9 of 16 references.
References · 9
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
- 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 783
- 2022Open-Vocabulary Semantic Segmentation with Mask-adapted CLIPcited by 565
- 2026Scaling Verification Can Be More Effective than Scaling Policy Learning for Vision-Language-Action Alignmentcited by 21
- 2025ZINA: Multimodal Fine-grained Hallucination Detection and Editingcited by 5
- 2025HalDec-Bench: Benchmarking Hallucination Detector in Image Captioningcited by 5
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025GroundingME: Exposing the Visual Grounding Gap in MLLMs through Multi-Dimensional Evaluation
- 2023Mismatch Quest: Visual and Textual Feedback for Image-Text Misalignment
- 2023Improved Visual Grounding through Self-Consistent Explanations
- 2023Simple Token-Level Confidence Improves Caption Correctness
- 2026ReViCo: Unveiling the Limitations of VLMs in Visual Text Understanding via Error Correction
- 2025Your Large Vision-Language Model Only Needs A Few Attention Heads For Visual Grounding
- 2024Q-GroundCAM: Quantifying Grounding in Vision Language Models via GradCAM
- 2026PANORAMA: Panoptic Grounded Captioning via Mask Proposal Selection