CDH-Bench: A Commonsense-Driven Hallucination Benchmark for Evaluating Visual Fidelity in Vision-Language Models

Vision-language models (VLMs) achieve strong performance on many benchmarks, yet a basic reliability question remains underexplored: when visual evidence conflicts with commonsense, do models follow what is shown or what commonsense suggests? A characteristic failure in this setting is that the model overrides visual evidence and outputs the commonsense alternative. We term this phenomenon commonsense-driven hallucination (CDH). To evaluate it, we introduce CDH-Bench, a benchmark designed to create explicit visual evidence--commonsense conflicts. CDH-Bench covers three dimensions: textitcounting anomalies, textitrelational anomalies, and textitattribute anomalies. We evaluate frontier VLMs under textitbinary Question Answering (QA) and textitmultiple-choice QA, and report metrics including textitCounterfactual Accuracy (CF-Acc), textitCommonsense Accuracy (CS-Acc), textitCounterfactual Accuracy Drop (CFAD), textitCommonsense Collapse Rate (CCR), and textitRelative Prior Dependency (RPD). Results show that even strong models remain vulnerable to prior-driven normalization under visual evidence--commonsense conflict. CDH-Bench provides a controlled diagnostic of visual fidelity under visual evidence--commonsense conflict.