# H-POPE: Hierarchical Polling-based Probing Evaluation of Hallucinations in Large Vision-Language Models

arXiv: https://arxiv.org/abs/2411.04077 · HTML: https://mlsanity.com/paper/2411.04077/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 7 of its 9 references that are in the map.
- First posted on arXiv 6 Nov 2024.

## Placed in

- Multimodal LLMs / vision-language models / [Hallucination mitigation in LVLMs](https://mlsanity.com/mllm/hallucination-mitigation-in-lvlms/)

## Builds on (most cited first, 7 of 7)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [Evaluating Object Hallucination in Large Vision-Language Models](https://mlsanity.com/paper/2305.10355/) (2023)
- [Object Hallucination in Image Captioning](https://mlsanity.com/paper/1809.02156/) (2018)
- [Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers](https://mlsanity.com/paper/2103.15679/) (2021)
- [Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directions](https://mlsanity.com/paper/2404.07214/) (2024)

## Built on by (most cited first, 6 of 6)

- [Evaluating Hallucination in Large Vision-Language Models based on Context-Aware Object Similarities](https://mlsanity.com/paper/2501.15046/) (2025)
- [Seeing without Looking: Do Vision-Language Benchmarks Really Test Vision?](https://mlsanity.com/paper/2605.22903/) (2026)
- [HEDGE: Hallucination Estimation via Dense Geometric Entropy for VQA with Vision-Language Models](https://mlsanity.com/paper/2511.12693/) (2025)
- [Can Humans Dream of Electric Sheep? Human-Written Samples for Fine-Grained Vision-and-Language Hallucination Benchmarking](https://mlsanity.com/paper/2608.01021/) (2026)
- [RCP: Representation Consistency Pruner for Mitigating Distribution Shift in Large Vision-Language Models](https://mlsanity.com/paper/2604.04972/) (2026)
- [What Makes "Good" Distractors for Object Hallucination Evaluation in Large Vision-Language Models?](https://mlsanity.com/paper/2508.06530/) (2025)

## Abstract (arXiv)

By leveraging both texts and images, large vision language models (LVLMs) have shown significant progress in various multi-modal tasks. Nevertheless, these models often suffer from hallucinations, e.g., they exhibit inconsistencies between the visual input and the textual output. To address this, we propose H-POPE, a coarse-to-fine-grained benchmark that systematically assesses hallucination in object existence and attributes. Our evaluation shows that models are prone to hallucinations on object existence, and even more so on fine-grained attributes. We further investigate whether these models rely on visual input to formulate the output texts.