mlsanityData to 14 Oct 2026

A Survey of Multimodal Hallucination Evaluation and Detection

arXiv 2507.19024 ↗First posted 25 Jul 2025cs.CVcited by 27

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Multi-modal Large Language Models (MLLMs) have emerged as a powerful paradigm for integrating visual and textual information, supporting a wide range of multi-modal tasks. However, these models often suffer from hallucination, producing content that appears plausible but contradicts the input content or established world knowledge. This survey offers an in-depth review of hallucination evaluation benchmarks and detection methods across Image-to-Text (I2T) and Text-to-image (T2I) generation tasks. Specifically, we first propose a taxonomy of hallucination based on faithfulness and factuality, incorporating the common types of hallucinations observed in practice. Then we provide an overview of existing hallucination evaluation benchmarks for both T2I and I2T tasks, highlighting their construction process, evaluation objectives, and employed metrics. Furthermore, we summarize recent advances in hallucination detection methods, which aims to identify hallucinated content at the instance level and serve as a practical complement of benchmark-based evaluation. Finally, we highlight key limitations in current benchmarks and detection methods, and outline potential directions for future research.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 184 of its 245 references that are in the map. First posted on arXiv 25 Jul 2025.

Builds on

184 of 245 references.

Closest ancestors · 46

Same direction, newest first

  1. 2025HalLoc: Token-level Localization of Hallucinations for Vision Language Models12 Jun
  2. 2025Mitigating Hallucinations in Large Vision-Language Models via Entity-Centric Multimodal Preference Optimization4 Jun
  3. 2025Antidote: A Unified Framework for Mitigating LVLM Hallucinations in Counterfactual Presupposition and Object Perception29 Apr
  4. 2025TARAC: Mitigating Hallucination in LVLMs via Temporal Attention Real-time Accumulative Connection5 Apr
  5. 2025Treble Counterfactual VLMs: A Causal Approach to Hallucination8 Mar
  6. 2025MedHallTune: An Instruction-Tuning Benchmark for Mitigating Medical Hallucination in Vision-Language Models28 Feb
  7. 2025CutPaste&Find: Efficient Multimodal Hallucination Detector with Visual-aid Knowledge Base18 Feb
  8. 2024Cracking the Code of Hallucination in LVLMs with Vision-aware Head Divergence18 Dec
  9. 2024DHCP: Detecting Hallucinations by Cross-modal Attention Pattern in Large Vision-Language Models27 Nov
  10. 2024VL-Uncertainty: Detecting Hallucination in Large Vision-Language Model via Uncertainty Estimation18 Nov
  11. 2024V-DPO: Mitigating Hallucination in Large Vision Language Models via Vision-Guided Direct Preference Optimization5 Nov
  12. 2024LongHalQA: Long-Context Hallucination Evaluation for MultiModal Large Language Models13 Oct
34 more
  1. 2024Interpreting and Editing Vision-Language Representations to Mitigate Hallucinations3 Oct
  2. 2024ODE: Open-Set Evaluation of Hallucinations in Multimodal Large Language Models14 Sep
  3. 2024Reference-free Hallucination Detection for Large Vision-Language Models11 Aug
  4. 2024Multi-Object Hallucination in Vision-Language Models8 Jul
  5. 2024Measuring the Measurers: Quality Evaluation of Hallucination Benchmarks for Large Vision-Language Models24 Jun
  6. 2024Evaluating and Analyzing Relationship Hallucinations in Large Vision-Language Models24 Jun
  7. 2024AutoHallusion: Automatic Generation of Hallucination Benchmarks for Vision-Language Models16 Jun
  8. 2024Detecting and Evaluating Medical Hallucinations in Large Vision Language Models14 Jun
  9. 2024THRONE: An Object-based Hallucination Benchmark for the Free-form Generations of Large Vision-Language Models8 May
  10. 2024Hallucination of Multimodal Large Language Models: A Survey29 Apr
  11. 2024Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback22 Apr
  12. 2024PhD: A ChatGPT-Prompted Visual hallucination Evaluation Dataset17 Mar
  13. 2024Hal-Eval: A Universal and Fine-grained Hallucination Evaluation Framework for Large Vision Language Models24 Feb
  14. 2024Visual Hallucinations of Multi-modal Large Language Models22 Feb
  15. 2024Unified Hallucination Detection for Multimodal Large Language Models5 Feb
  16. 2024A Survey on Hallucination in Large Vision-Language Models1 Feb
  17. 2023Mitigating Fine-Grained Hallucination by Fine-Tuning Large Vision-Language Models with Caption Rewrites4 Dec
  18. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation29 Nov
  19. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
  20. 2023Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimization28 Nov
  21. 2023Mitigating Hallucination in Visual Language Models with Visual Supervision27 Nov
  22. 2023HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Data22 Nov
  23. 2023AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation13 Nov
  24. 2023FaithScore: Fine-grained Evaluations of Hallucinations in Large Vision-Language Models2 Nov
  25. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models23 Oct
  26. 2023Negative Object Presence Evaluation (NOPE) to Measure Object Hallucination in Vision-Language Models9 Oct
  27. 2023HallE-Control: Controlling Object Hallucination in Large Multimodal Models3 Oct
  28. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Models1 Oct
  29. 2023CIEM: Contrastive Instruction Evaluation Method for Better Instruction Tuning5 Sep
  30. 2023Evaluation and Analysis of Hallucination in Large Vision-Language Models29 Aug
  31. 2023Detecting and Preventing Hallucinations in Large Vision Language Models11 Aug
  32. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning26 Jun
  33. 2023Evaluating Object Hallucination in Large Vision-Language Models17 May
  34. 2022Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-training14 Oct

Other references · 138

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  8. 2023Visual Instruction Tuningcited by 7,869
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  10. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  11. 2022Scalable Diffusion Models with Transformerscited by 6,054
  12. 2022Flow Matching for Generative Modelingcited by 5,702
38 more
  1. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  2. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  3. 2017Neural Discrete Representation Learningcited by 4,712
  4. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  5. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  6. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  7. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  8. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  9. 2017Deep reinforcement learning from human preferencescited by 3,912
  10. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  11. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  12. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  13. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  14. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  15. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  16. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  17. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  18. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  19. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  20. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  21. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  22. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  23. 2020Center-based 3D Object Detection and Trackingcited by 1,144
  24. 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 972
  25. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
  26. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 781
  27. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 735
  28. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 712
  29. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 702
  30. 2023Baichuan 2: Open Large-scale Language Modelscited by 670
  31. 2018Object Hallucination in Image Captioningcited by 669
  32. 2018Generative Image Inpainting with Contextual Attentioncited by 580
  33. 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 561
  34. 2018The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scalecited by 544
  35. 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 474
  36. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 451
  37. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 449
  38. 2018nocaps: novel object captioning at scalecited by 444

50 most cited of 138 All 138 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 24.

27 papers.

Sort:

By area:

  1. 2026Unify-Agent: A Unified Multimodal Agent for World-Grounded Image Synthesiscited by 17
  2. 2026Dual-Pathway Circuits of Object Hallucination in Vision-Language Modelscited by 7
  3. 2025Journey Before Destination: On the importance of Visual Faithfulness in Slow Thinkingcited by 6
  4. 2025HEDGE: Hallucination Estimation via Dense Geometric Entropy for VQA with Vision-Language Modelscited by 5
  5. 2026Residual Decoding: Mitigating Hallucinations in Large Vision-Language Models via History-Aware Residual Guidancecited by 5
  6. 2026FloorplanVLM: A Vision-Language Model for Floorplan Vectorizationcited by 5
  7. 2026Automatic Method Illustration Generation for AI Scientific Papers via Drawing Middleware Creation, Evolution, and Orchestrationcited by 5
  8. 2026Fanar-Sadiq: A Multi-Agent Architecture for Grounded Islamic QAcited by 3
  9. 2026Once Correct, Still Wrong: Counterfactual Hallucination in Multilingual Vision-Language Modelscited by 2
  10. 2026ViSA-Enhanced Aerial VLN: A Visual-Spatial Reasoning Enhanced Framework for Aerial Vision-Language Navigationcited by 2
  11. 2026KnowHal: A Knowledge-Driven Benchmark for Comprehensive Multimodal Hallucination Evaluationcited by 2
  12. 2026FaithSCAN: Model-Driven Single-Pass Hallucination Detection for Faithful Visual Question Answeringcited by 1
15 more
  1. 2026VideoHEDGE: Entropy-Based Hallucination Detection for Video-VLMs via Semantic Clustering and Spatiotemporal Perturbationscited by 1
  2. 2026Now You See the Hate: Adaptive View Retrieval for Hidden Hateful Illusionscited by 1
  3. 2025Hallucination as an Upper Bound: A New Perspective on Text-to-Image Evaluationcited by 0
  4. 2026TASTE: A Designer-Annotated Multi-Dimensional Preference Dataset for AI-Generated Graphic Designcited by 0
  5. 2026Hallo4D: Multi-Modal Hallucination Mitigation for Consistent Spatio-Temporal Generationcited by 0
  6. 2026HalluScope: Fine-grained Hallucination Diagnosis for Multimodal Large Language Modelscited by 0
  7. 2026Witness Evidence Portfolios: Single-Prefill Risk Detection for Closed Multimodal Answerscited by 0
  8. 2026Uncertainty-Aware Decision Making in Multimodal Large Language Modelscited by 0
  9. 2026Hallucination Mitigation for Large Vision-Language Models via Implicit Feature Stabilizationcited by 0
  10. 2026ImageEval 2026: Culturally Grounded Arabic Multimodal Evaluationcited by 0
  11. 2026MLLMs Hallucinate when Information Distribution Drifts in Synergy Headscited by 0
  12. 2026Dissecting Training-Free Uncertainty Estimation in Multimodal Large Language Modelscited by 0
  13. 2026AdaPilot: Towards Scene-Adaptive Policy Learning for Cross-Generator Text-to-Image Quality Optimizationcited by 0
  14. 2026From Pixel Generation to Topological Inference: Structural Dual Super-Resolution for Trustworthy Cross-Physical-Domain Trabecular Morphology Learningcited by 0
  15. 2026SAGE: Sink-Aware Guided Emphasis for Visual Grounding in Vision-Language Decoderscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025MIHBench: Benchmarking and Mitigating Multi-Image Hallucinations in Multimodal Large Language Models
  2. 2026OmniHallu: Unified Hallucination Detection for Cross-Modal Comprehension and Generation in Multimodal Large Language Models
  3. 2025Mitigating Behavioral Hallucination in Multimodal Large Language Models for Sequential Images
  4. 2026ReactBench: A Cause-Driven Benchmark for Multimodal Hallucination via Systematic Evaluation
  5. 2025HalluSearch at SemEval-2025 Task 3: A Search-Enhanced RAG Pipeline for Hallucination Detection
  6. 2023HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models
  7. 2026FREAK: A Fine-grained Hallucination Evaluation Benchmark for Advanced MLLMs
  8. 2024VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Models