# VIGIL: Tackling Hallucination Detection in Image Recontextualization

arXiv: https://arxiv.org/abs/2602.14633 · HTML: https://mlsanity.com/paper/2602.14633/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 29 of its 33 references that are in the map.
- First posted on arXiv 16 Feb 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Hallucination mitigation in LVLMs](https://mlsanity.com/mllm/hallucination-mitigation-in-lvlms/)

## Builds on (most cited first, 29 of 29)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Evaluating Object Hallucination in Large Vision-Language Models](https://mlsanity.com/paper/2305.10355/) (2023)
- [FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space](https://mlsanity.com/paper/2506.15742/) (2025)
- [SAM 3: Segment Anything with Concepts](https://mlsanity.com/paper/2511.16719/) (2025)
- [Object Hallucination in Image Captioning](https://mlsanity.com/paper/1809.02156/) (2018)
- [Analyzing and Mitigating Object Hallucination in Large Vision-Language Models](https://mlsanity.com/paper/2310.00754/) (2023)
- [AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation](https://mlsanity.com/paper/2311.07397/) (2023)
- [Woodpecker: Hallucination Correction for Multimodal Large Language Models](https://mlsanity.com/paper/2310.16045/) (2023)
- [WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generation](https://mlsanity.com/paper/2503.07265/) (2025)
- [VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluation](https://mlsanity.com/paper/2312.14867/) (2023)
- [Dress Code: High-Resolution Multi-Category Virtual Try-On](https://mlsanity.com/paper/2204.08532/) (2022)
- [GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generation](https://mlsanity.com/paper/2406.13743/) (2024)
- [Understanding Hallucinations in Diffusion Models through Mode Interpolation](https://mlsanity.com/paper/2406.09358/) (2024)
- [ImagenHub: Standardizing the evaluation of conditional image generation models](https://mlsanity.com/paper/2310.01596/) (2023)
- [LEGION: Learning to Ground and Explain for Synthetic Image Detection](https://mlsanity.com/paper/2503.15264/) (2025)
- [Hallucination Detection in LLMs Using Spectral Features of Attention Maps](https://mlsanity.com/paper/2502.17598/) (2025)
- [Evaluating Image Hallucination in Text-to-Image Generation with Question-Answering](https://mlsanity.com/paper/2409.12784/) (2024)
- [A Unified Agentic Framework for Evaluating Conditional Image Generation](https://mlsanity.com/paper/2504.07046/) (2025)
- [Evaluating Hallucination in Text-to-Image Diffusion Models with Scene-Graph based Question-Answering Agent](https://mlsanity.com/paper/2412.05722/) (2024)
- [Explainable Synthetic Image Detection through Diffusion Timestep Ensembling](https://mlsanity.com/paper/2503.06201/) (2025)
- [Mitigating Diffusion Model Hallucinations with Dynamic Guidance](https://mlsanity.com/paper/2510.05356/) (2025)
- [HalDec-Bench: Benchmarking Hallucination Detector in Image Captioning](https://mlsanity.com/paper/2511.20515/) (2025)
- [KEA Explain: Explanations of Hallucinations using Graph Kernel Analysis](https://mlsanity.com/paper/2507.03847/) (2025)
- [REVEALER: Reinforcement-Guided Visual Reasoning for Element-Level Text-Image Alignment Evaluation](https://mlsanity.com/paper/2512.23169/) (2025)
- [Make VLM Recognize Visual Hallucination on Cartoon Character Image with Pose Information](https://mlsanity.com/paper/2403.15048/) (2024)

## Abstract (arXiv)

We introduce VIGIL (Visual Inconsistency & Generative In-context Lucidity), a benchmark dataset and framework that provides a fine-grained categorization of hallucinations in the multimodal image recontextualization task for large multimodal models. Most existing methods treat hallucinations as a single undifferentiated error. We instead decompose them into five categories, namely Object Visual Fidelity, Background Fidelity, Spatial and Instructional Fidelity, Physical and Integration Fidelity, and Object Omission. We propose a multi-stage detection pipeline that processes recontextualized images through specialized steps targeting all five categories with a coordinated set of open-source models. We evaluate the pipeline on the VIGIL dataset of 1,269 manually annotated samples across five product domains and report macro-F1 per category. The decomposed pipeline reaches the best macro-F1 among open-source detectors. It returns a textual explanation for each detected error, which prior methods for this task do not provide. We openly release VIGIL, along with the detection pipeline and benchmark code, through our GitHub repository: https://github.com/mlubneuskaya/vigil and Data repository: https://huggingface.co/datasets/joannaww/VIGIL.