mlsanityData to 7 Oct 2026

A Survey on Hallucination in Large Vision-Language Models

arXiv 2402.00253 ↗First posted 1 Feb 2024cs.CV cs.CL cs.LGcited by 352

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Recent development of Large Vision-Language Models (LVLMs) has attracted growing attention within the AI landscape for its practical implementation potential. However, ``hallucination'', or more specifically, the misalignment between factual visual content and corresponding textual generation, poses a significant challenge of utilizing LVLMs. In this comprehensive survey, we dissect LVLM-related hallucinations in an attempt to establish an overview and facilitate future mitigation. Our scrutiny starts with a clarification of the concept of hallucinations in LVLMs, presenting a variety of hallucination symptoms and highlighting the unique challenges inherent in LVLM hallucinations. Subsequently, we outline the benchmarks and methodologies tailored specifically for evaluating hallucinations unique to LVLMs. Additionally, we delve into an investigation of the root causes of these hallucinations, encompassing insights from the training data and model components. We also critically review existing methods for mitigating hallucinations. The open questions and future directions pertaining to hallucinations within LVLMs are discussed to conclude this survey.

Key facts. Cited by 352 papers in the map (data through 7 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 53 of its 59 references that are in the map. First posted on arXiv 1 Feb 2024.

Builds on

53 of 59 references.

Closest ancestors · 15

Same direction, newest first

  1. 2023Hallucination Augmented Contrastive Learning for Multimodal Large Language Model12 Dec
  2. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback1 Dec
  3. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation29 Nov
  4. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
  5. 2023Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimization28 Nov
  6. 2023AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation13 Nov
  7. 2023Volcano: Mitigating Multimodal Hallucination through Self-Feedback Guided Revision13 Nov
  8. 2023Negative Object Presence Evaluation (NOPE) to Measure Object Hallucination in Vision-Language Models9 Oct
  9. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Models1 Oct
  10. 2023CIEM: Contrastive Instruction Evaluation Method for Better Instruction Tuning5 Sep
  11. 2023Evaluation and Analysis of Hallucination in Large Vision-Language Models29 Aug
  12. 2023Detecting and Preventing Hallucinations in Large Vision Language Models11 Aug
3 more
  1. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning26 Jun
  2. 2023Evaluating Object Hallucination in Large Vision-Language Models17 May
  3. 2022Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-training14 Oct

Other references · 38

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2023Visual Instruction Tuningcited by 7,792
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  8. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  9. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  10. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  11. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  12. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
26 more
  1. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  2. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  3. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  4. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
  5. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  6. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
  7. 2023A Survey on Multimodal Large Language Modelscited by 898
  8. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
  9. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  10. 2018Object Hallucination in Image Captioningcited by 661
  11. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 617
  12. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  13. 2023Ferret: Refer and Ground Anything Anywhere at Any Granularitycited by 498
  14. 2023Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Modelscited by 356
  15. 2023Recognize Anything: A Strong Image Tagging Modelcited by 305
  16. 2023LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agentscited by 188
  17. 2023ImageBind-LLM: Multi-modality Instruction Tuningcited by 142
  18. 2018Neural Baby Talkcited by 107
  19. 2021Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Groundingcited by 106
  20. 2023VIGC: Visual Instruction Generation and Correctioncited by 80
  21. 2022Fine-grained Image Captioning with CLIP Rewardcited by 69
  22. 2023Position-Enhanced Visual Instruction Tuning for Multimodal Large Language Modelscited by 59
  23. 2022Vision-and-Language Pretrained Models: A Surveycited by 50
  24. 2023Can We Edit Multimodal Large Language Models?cited by 44
  25. 2023VCoder: Versatile Vision Encoders for Multimodal Large Language Modelscited by 42
  26. 2023Enhancing the Spatial Awareness Capability of Multi-Modal Large Language Modelcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 84; 2025: 147; 2026: 121.

352 papers.

Sort:

By area:

  1. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467
  2. 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 193
  3. 2024Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMscited by 174
  4. 2025OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Modelcited by 172
  5. 2024Vision-Language Models for Medical Report Generation and Visual Question Answering: A Reviewcited by 121
  6. 2025MMSearch-R1: Incentivizing LMMs to Searchcited by 120
  7. 2024IBD: Alleviating Hallucinations in Large Vision-Language Models via Image-Biased Decodingcited by 118
  8. 2024VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image Understandingcited by 116
  9. 2024AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Promptingcited by 113
  10. 2025Self-Rewarding Vision-Language Model via Reasoning Decompositioncited by 95
  11. 2024MLLM can see? Dynamic Correction Decoding for Hallucination Mitigationcited by 87
  12. 2025NoisyRollout: Reinforcing Visual Reasoning with Data Augmentationcited by 86
38 more
  1. 2024Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local Attentioncited by 74
  2. 2024CoVLA: Comprehensive Vision-Language-Action Dataset for Autonomous Drivingcited by 74
  3. 2024Vript: A Video Is Worth Thousands of Wordscited by 71
  4. 2024VL-Uncertainty: Detecting Hallucination in Large Vision-Language Model via Uncertainty Estimationcited by 71
  5. 2024Visual Hallucinations of Multi-modal Large Language Modelscited by 68
  6. 2024BRAVE: Broadening the visual encoding of vision-language modelscited by 68
  7. 2024Benchmarking Multimodal Retrieval Augmented Generation with Dynamic VQA Dataset and Self-adaptive Planning Agentcited by 65
  8. 2025Mitigating Hallucinations in Large Vision-Language Models via DPO: On-Policy Data Hold the Keycited by 64
  9. 2025The Hidden Life of Tokens: Reducing Hallucination of Large Vision-Language Models via Visual Information Steeringcited by 62
  10. 2024Visual-RolePlay: Universal Jailbreak Attack on MultiModal Large Language Models via Role-playing Image Charactercited by 61
  11. 2024Reducing Hallucinations in Vision-Language Models via Latent Space Steeringcited by 56
  12. 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Surveycited by 52
  13. 2024MultiTrust: A Comprehensive Benchmark Towards Trustworthy Multimodal Large Language Modelscited by 51
  14. 2024Doe-1: Closed-Loop Autonomous Driving with Large World Modelcited by 51
  15. 2024Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Surveycited by 50
  16. 2024AVHBench: A Cross-Modal Hallucination Benchmark for Audio-Visual Large Language Modelscited by 50
  17. 2024Mitigating Object Hallucination via Concentric Causal Attentioncited by 48
  18. 2025UAVs Meet LLMs: Overviews and Perspectives Toward Agentic Low-Altitude Mobilitycited by 48
  19. 2025A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challengescited by 46
  20. 2025Self-Correcting Decoding with Generative Feedback for Mitigating Hallucinations in Large Vision-Language Modelscited by 44
  21. 2024Mitigating Modality Prior-Induced Hallucinations in Multimodal Large Language Models via Deciphering Attention Causalitycited by 42
  22. 2025Seeing Far and Clearly: Mitigating Hallucinations in MLLMs with Attention Causal Decodingcited by 42
  23. 2024Paint by Inpaint: Learning to Add Image Objects by Removing Them Firstcited by 39
  24. 2024CODE: Contrasting Self-generated Description to Combat Hallucination in Large Multi-modal Modelscited by 37
  25. 2025Visual Large Language Models for Generalized and Specialized Applicationscited by 35
  26. 2025Reasoning about Uncertainty: Do Reasoning Models Know When They Don't Know?cited by 35
  27. 2024Visual Description Grounding Reduces Hallucinations and Boosts Reasoning in LVLMscited by 33
  28. 2025Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoningcited by 33
  29. 2025MM-CoT:A Benchmark for Probing Visual Chain-of-Thought Reasoning in Multimodal Modelscited by 33
  30. 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 32
  31. 2025CHiP: Cross-modal Hierarchical Direct Preference Optimization for Multimodal LLMscited by 31
  32. 2024AutoHallusion: Automatic Generation of Hallucination Benchmarks for Vision-Language Modelscited by 30
  33. 2024HaloQuest: A Visual Hallucination Dataset for Advancing Multimodal Reasoningcited by 30
  34. 2025Mem2Ego: Empowering Vision-Language Models with Global-to-Ego Memory for Long-Horizon Embodied Navigationcited by 30
  35. 2024Automated Multi-level Preference for MLLMscited by 29
  36. 2024Can Large Audio-Language Models Truly Hear? Tackling Hallucinations with Multi-Task Assessment and Stepwise Audio Reasoningcited by 29
  37. 2024HumanEdit: A High-Quality Human-Rewarded Dataset for Instruction-based Image Editingcited by 29
  38. 2024ICT: Image-Object Cross-Level Trusted Intervention for Mitigating Object Hallucination in Large Vision-Language Modelscited by 28

50 most cited of 352 All 352 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Hallucination in Multimodal Foundation Models: A Survey on Causes, Corrections, and Evaluations
  2. 2025A Comprehensive Analysis for Visual Object Hallucination in Large Vision-Language Models
  3. 2024Hallucination of Multimodal Large Language Models: A Survey
  4. 2024From Pixels to Tokens: Revisiting Object Hallucinations in Large Vision-Language Models
  5. 2024Hal-Eval: A Universal and Fine-grained Hallucination Evaluation Framework for Large Vision Language Models
  6. 2026What Makes LVLMs Hallucinate Less? Unveiling the Architectural Factors Behind Hallucination Robustness
  7. 2026Reducing Object Hallucination in LVLMs via Emphasizing Image-negative Tokens
  8. 2024Cracking the Code of Hallucination in LVLMs with Vision-aware Head Divergence