DO-Bench: An Attributable Benchmark for Diagnosing Object Hallucination in Vision-Language Models
On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs
Abstract (arXiv)
Object level hallucination remains a central reliability challenge for vision language models (VLMs), particularly in binary object existence verification. Existing benchmarks emphasize aggregate accuracy but rarely disentangle whether errors stem from perceptual limitations or from the influence of contextual textual priors, leaving underlying failure mechanisms ambiguous. We introduce DO-Bench, a controlled diagnostic benchmark that isolates these sources through structured multimodal interventions. Rather than evaluating models in unconstrained settings, DO-Bench probes two complementary dimensions: the Prior Override dimension progressively strengthens contextual textual priors while holding visual evidence constant to assess resistance to prior pressure, and the Perception-Limited dimension incrementally enhances visual evidence from full-scene context to localized object crops to measure perceptual grounding strength. This paired design enables attribution of errors to prior suppression, perceptual insufficiency, or their interaction. We further define two diagnostic metrics, PriorRobust and PerceptionAbility, to quantify these behaviors consistently. Evaluations across diverse open- and closed-source VLMs reveal systematic differences in prior sensitivity and perceptual reliability, demonstrating that object hallucination reflects heterogeneous, mechanism dependent failure patterns beyond aggregate accuracy.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 33 of its 43 references that are in the map. First posted on arXiv 18 Apr 2026.
Builds on
33 of 43 references.
Closest ancestors · 16
Same direction, newest first
- 2025Mitigating Hallucinations in Large Vision-Language Models by Self-Injecting Hallucinations14 Sep
- 2025Steering LVLMs via Sparse Autoencoder for Hallucination Mitigation22 May
- 2025Mitigating Image Captioning Hallucinations in Vision-Language Models6 May
- 2025Mitigating Object Hallucinations in MLLMs via Multi-Frequency Perturbations19 Mar
- 2024From Pixels to Tokens: Revisiting Object Hallucinations in Large Vision-Language Models9 Oct
- 2024Multi-Object Hallucination in Vision-Language Models8 Jul
- 2024THRONE: An Object-based Hallucination Benchmark for the Free-form Generations of Large Vision-Language Models8 May
- 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation29 Nov
- 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
- 2023AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation13 Nov
- 2023Woodpecker: Hallucination Correction for Multimodal Large Language Models24 Oct
- 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models23 Oct
4 more
- 2023Negative Object Presence Evaluation (NOPE) to Measure Object Hallucination in Vision-Language Models9 Oct
- 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Models1 Oct
- 2023Detecting and Preventing Hallucinations in Large Vision Language Models11 Aug
- 2022Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-training14 Oct
Other references · 17
- 2023Visual Instruction Tuningcited by 7,792
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
- 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
- 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
- 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 367
- 2021VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomenacited by 134
- 2022VL-CheckList: Evaluating Pre-trained Vision-Language Models with Objects, Attributes and Relationscited by 113
- 2024Effectiveness Assessment of Recent Large Vision-Language Modelscited by 29
- 2023Can Linguistic Knowledge Improve Multimodal Alignment in Vision-Language Pretraining?cited by 21
5 more
- 2025Vision-Language Model for Object Detection and Segmentation: A Review and Evaluationcited by 10
- 2025Why 1 + 1 < 1 in Visual Token Pruning: Beyond Naive Integration via Multi-Objective Balanced Coveringcited by 10
- 2024HumanVLM: Foundation for Human-Scene Vision-Language Modelcited by 8
- 2026DeepScan: A Training-Free Framework for Visually Grounded Reasoning in Large Vision-Language Modelscited by 6
- 2025Exploring the Secondary Risks of Large Language Modelscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Vision-Language Introspection: Mitigating Overconfident Hallucinations in MLLMs via Interpretable Bi-Causal Steering
- 2026VisER: Visual Evidence and Reliance for Object Hallucination Detection in LVLMs
- 2026What Do Hallucinations Reveal About Multimodal Reasoning? Diagnosing Visual Grounding Failures via Contrastive Decoding Probes
- 2024H-POPE: Hierarchical Polling-based Probing Evaluation of Hallucinations in Large Vision-Language Models
- 2026Fine-Grained Multi Image Object Hallucination Benchmark
- 2025Generate, but Verify: Reducing Hallucination in Vision-Language Models with Retrospective Resampling
- 2026LLM-as-Judge Framework for Evaluating Tone-Induced Hallucination in Vision-Language Models
- 2025Analyzing and Mitigating Object Hallucination: A Training Bias Perspective