mlsanityData to 7 Oct 2026

Detecting and Preventing Hallucinations in Large Vision Language Models

arXiv 2308.06394 ↗First posted 11 Aug 2023cs.CV cs.LGcited by 254

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

also in 1 more
Abstract (arXiv)

Instruction tuned Large Vision Language Models (LVLMs) have significantly advanced in generalizing across a diverse set of multi-modal tasks, especially for Visual Question Answering (VQA). However, generating detailed responses that are visually grounded is still a challenging task for these models. We find that even the current state-of-the-art LVLMs (InstructBLIP) still contain a staggering 30 percent of the hallucinatory text in the form of non-existent objects, unfaithful descriptions, and inaccurate relationships. To address this, we introduce M-HalDetect, a (M)ultimodal (Hal)lucination (Detect)ion Dataset that can be used to train and benchmark models for hallucination detection and prevention. M-HalDetect consists of 16k fine-grained annotations on VQA examples, making it the first comprehensive multi-modal hallucination detection dataset for detailed image descriptions. Unlike previous work that only consider object hallucination, we additionally annotate both entity descriptions and relationships that are unfaithful. To demonstrate the potential of this dataset for hallucination prevention, we optimize InstructBLIP through our novel Fine-grained Direct Preference Optimization (FDPO). We also train fine-grained multi-modal reward models from InstructBLIP and evaluate their effectiveness with best-of-n rejection sampling. We perform human evaluation on both FDPO and rejection sampling, and find that they reduce hallucination rates in InstructBLIP by 41% and 55% respectively. We also find that our reward model generalizes to other multi-modal models, reducing hallucinations in LLaVA and mPLUG-OWL by 15% and 57% respectively, and has strong correlation with human evaluated accuracy scores.

Key facts. Cited by 254 papers in the map (data through 7 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 27 of its 36 references that are in the map. First posted on arXiv 11 Aug 2023.

Builds on

27 of 36 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning26 Jun
  2. 2023Evaluating Object Hallucination in Large Vision-Language Models17 May

Other references · 25

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2017Proximal Policy Optimization Algorithmscited by 14,441
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  8. 2023Visual Instruction Tuningcited by 7,792
  9. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  10. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  11. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  12. 2023Let's Verify Step by Stepcited by 3,598
13 more
  1. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  2. 2020Learning to summarize from human feedbackcited by 2,562
  3. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  4. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  5. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  6. 2023Large Language Models are not Fair Evaluatorscited by 1,050
  7. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  8. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  9. 2023Fine-Grained Human Feedback Gives Better Rewards for Language Model Trainingcited by 399
  10. 2023Secrets of RLHF in Large Language Models Part I: PPOcited by 226
  11. 2020Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPOcited by 184
  12. 2018ROUGE 2.0: Updated and Improved Measures for Evaluation of Summarization Taskscited by 66
  13. 2023Large Multimodal Models: Notes on CVPR 2023 Tutorialcited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 16; 2024: 83; 2025: 100; 2026: 55.

254 papers.

Sort:

By area:

  1. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  2. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 671
  3. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 572
  4. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedbackcited by 404
  5. 2023A Survey of Hallucination in Large Foundation Modelscited by 365
  6. 2024MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmarkcited by 362
  7. 2024A Survey on Hallucination in Large Vision-Language Modelscited by 352
  8. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Modelscited by 314
  9. 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 296
  10. 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoningcited by 267
  11. 2023Multimodal Foundation Models: From Specialists to General-Purpose Assistantscited by 247
  12. 2023Woodpecker: Hallucination Correction for Multimodal Large Language Modelscited by 223
38 more
  1. 2023Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimizationcited by 223
  2. 2025Unified Reward Model for Multimodal Understanding and Generationcited by 194
  3. 2024Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaborationcited by 187
  4. 2024Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMscited by 174
  5. 2024HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decodingcited by 171
  6. 2024Direct Preference Optimization of Video Large Multimodal Models from Language Model Rewardcited by 145
  7. 2024Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lenscited by 139
  8. 2024Improve Vision Language Model Chain-of-thought Reasoningcited by 135
  9. 2024ShapeLLM: Universal 3D Object Understanding for Embodied Interactioncited by 133
  10. 2023Holistic Analysis of Hallucination in GPT-4V(ision): Bias and Interference Challengescited by 131
  11. 2024RLAIF-V: Open-Source AI Feedback Leads to Super GPT-4V Trustworthinesscited by 129
  12. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Modelscited by 129
  13. 2023HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Datacited by 124
  14. 2024Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Modelscited by 90
  15. 2024MLLM can see? Dynamic Correction Decoding for Hallucination Mitigationcited by 87
  16. 2024Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local Attentioncited by 74
  17. 2024Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Modelscited by 72
  18. 2024V-DPO: Mitigating Hallucination in Large Vision Language Models via Vision-Guided Direct Preference Optimizationcited by 67
  19. 2024CARES: A Comprehensive Benchmark of Trustworthiness in Medical Vision Language Modelscited by 64
  20. 2024On Softmax Direct Preference Optimization for Recommendationcited by 64
  21. 2024MIA-Bench: Towards Better Instruction Following Evaluation of Multimodal LLMscited by 62
  22. 2024Detecting and Evaluating Medical Hallucinations in Large Vision Language Modelscited by 57
  23. 2024Reducing Hallucinations in Vision-Language Models via Latent Space Steeringcited by 56
  24. 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 54
  25. 2024Visual Agents as Fast and Slow Thinkerscited by 53
  26. 2024AVHBench: A Cross-Modal Hallucination Benchmark for Audio-Visual Large Language Modelscited by 50
  27. 2024CLIP-DPO: Vision-Language Models as a Source of Preference for Fixing Hallucinations in LVLMscited by 49
  28. 2024How Easy is It to Fool Your Multimodal LLMs? An Empirical Analysis on Deceptive Promptscited by 47
  29. 2024Don't Miss the Forest for the Trees: Attentional Vision Calibration for Large Vision Language Modelscited by 46
  30. 2024A Survey on Evaluation of Multimodal Large Language Modelscited by 46
  31. 2024Nullu: Mitigating Object Hallucinations in Large Vision-Language Models via HalluSpace Projectioncited by 46
  32. 2025A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challengescited by 46
  33. 2024Mitigating Object Hallucination in Large Vision-Language Models via Image-Grounded Guidancecited by 44
  34. 2024PhD: A ChatGPT-Prompted Visual hallucination Evaluation Datasetcited by 44
  35. 2024VLind-Bench: Measuring Language Priors in Large Vision-Language Modelscited by 44
  36. 2025Self-Correcting Decoding with Generative Feedback for Mitigating Hallucinations in Large Vision-Language Modelscited by 44
  37. 2025Seeing Far and Clearly: Mitigating Hallucinations in MLLMs with Attention Causal Decodingcited by 42
  38. 2025ClearSight: Visual Signal Enhancement for Object Hallucination Mitigation in Multimodal Large language Modelscited by 40

50 most cited of 254 All 254 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Towards a Systematic Evaluation of Hallucinations in Large-Vision Language Models
  2. 2024Beyond Logit Lens: Contextual Embeddings for Robust Hallucination Detection & Grounding in VLMs
  3. 2025CutPaste&Find: Efficient Multimodal Hallucination Detector with Visual-aid Knowledge Base
  4. 2025Seeing is Believing: Rich-Context Hallucination Detection for MLLMs via Backward Visual Grounding
  5. 2025HEDGE: Hallucination Estimation via Dense Geometric Entropy for VQA with Vision-Language Models
  6. 2024Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback
  7. 2025Analyzing and Mitigating Object Hallucination: A Training Bias Perspective
  8. 2024Visual Hallucination: Definition, Quantification, and Prescriptive Remediations