mlsanityData to 14 Oct 2026

Volcano: Mitigating Multimodal Hallucination through Self-Feedback Guided Revision

arXiv 2311.07362 ↗First posted 13 Nov 2023cs.CL cs.CVcited by 98

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Large multimodal models suffer from multimodal hallucination, where they provide incorrect responses misaligned with the given visual information. Recent works have conjectured that one of the reasons behind multimodal hallucination is due to the vision encoder failing to ground on the image properly. To mitigate this issue, we propose a novel approach that leverages self-feedback as visual cues. Building on this approach, we introduce Volcano, a multimodal self-feedback guided revision model. Volcano generates natural language feedback to its initial response based on the provided visual information and utilizes this feedback to self-revise its initial response. Volcano effectively reduces multimodal hallucination and achieves state-of-the-art on MMHal-Bench, POPE, and GAVIE. It also improves on general multimodal abilities and outperforms previous models on MM-Vet and MMBench. Through qualitative analysis, we show that Volcano's feedback is properly grounded on the image than the initial response. This indicates that Volcano can provide itself with richer visual information through feedback generation, leading to self-correct hallucinations. We publicly release our model, data, and code at https://github.com/kaistAI/Volcano}{github.com/kaistAI/Volcano

Key facts. Cited by 98 papers in the map (data through 14 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 24 of its 46 references that are in the map. First posted on arXiv 13 Nov 2023.

Builds on

24 of 46 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning26 Jun

Other references · 23

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  4. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  5. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  6. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  7. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  8. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  9. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  10. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
  11. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
  12. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
11 more
  1. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 748
  2. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 712
  3. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 625
  4. 2023Prometheus: Inducing Fine-grained Evaluation Capability in Language Modelscited by 575
  5. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 550
  6. 2023Fine-Grained Human Feedback Gives Better Rewards for Language Model Trainingcited by 401
  7. 2022Generating Sequences by Learning to Self-Correctcited by 277
  8. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 238
  9. 2023RL4F: Generating Natural Language Feedback with Reinforcement Learning for Repairing Model Outputscited by 105
  10. 2023Shepherd: A Critic for Language Model Generationcited by 97
  11. 2021Let there be a clock on the beach: Reducing Object Hallucination in Image Captioningcited by 76

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 34; 2025: 43; 2026: 20.

98 papers.

Sort:

By area:

  1. 2024A Survey on Hallucination in Large Vision-Language Modelscited by 357
  2. 2024MM-LLMs: Recent Advances in MultiModal Large Language Modelscited by 278
  3. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 240
  4. 2024The Emerged Security and Privacy of LLM Agent: A Survey with Case Studiescited by 131
  5. 2024MLLM can see? Dynamic Correction Decoding for Hallucination Mitigationcited by 88
  6. 2024Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local Attentioncited by 74
  7. 2024VidHalluc: Evaluating Temporal Hallucinations in Multimodal Large Language Models for Video Understandingcited by 68
  8. 2024V-DPO: Mitigating Hallucination in Large Vision Language Models via Vision-Guided Direct Preference Optimizationcited by 67
  9. 2023Mitigating Fine-Grained Hallucination by Fine-Tuning Large Vision-Language Models with Caption Rewritescited by 61
  10. 2024Reducing Hallucinations in Vision-Language Models via Latent Space Steeringcited by 59
  11. 2024EventHallusion: Diagnosing Event Hallucinations in Video LLMscited by 57
  12. 2025LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directionscited by 50
38 more
  1. 2024Mitigating Object Hallucination via Concentric Causal Attentioncited by 49
  2. 2024Logical Closed Loop: Uncovering Object Hallucinations in Large Vision-Language Modelscited by 47
  3. 2024How Easy is It to Fool Your Multimodal LLMs? An Empirical Analysis on Deceptive Promptscited by 47
  4. 2024Don't Miss the Forest for the Trees: Attentional Vision Calibration for Large Vision Language Modelscited by 47
  5. 2025A Survey of Multimodal Retrieval-Augmented Generationcited by 37
  6. 2024LLM Self-Correction with DeCRIM: Decompose, Critique, and Refine for Enhanced Following of Instructions with Multiple Constraintscited by 35
  7. 2025Towards Trustworthy GUI Agents: A Surveycited by 34
  8. 2024HaloQuest: A Visual Hallucination Dataset for Advancing Multimodal Reasoningcited by 32
  9. 2024ICT: Image-Object Cross-Level Trusted Intervention for Mitigating Object Hallucination in Large Vision-Language Modelscited by 30
  10. 2025Large Vision-Language Model Alignment and Misalignment: A Survey Through the Lens of Explainabilitycited by 28
  11. 2025Mitigating Object Hallucinations via Sentence-Level Early Interventioncited by 28
  12. 2024Securing Large Language Models: Addressing Bias, Misinformation, and Prompt Attackscited by 26
  13. 2025Mitigating Object Hallucinations in Large Vision-Language Models via Attention Calibrationcited by 23
  14. 2025Attention Reallocation: Towards Zero-cost and Controllable Hallucination Mitigation of MLLMscited by 23
  15. 2025Why Reasoning Matters? A Survey of Advancements in Multimodal Reasoning (v1)cited by 23
  16. 2024Mitigating Hallucinations in Large Vision-Language Models via Summary-Guided Decodingcited by 22
  17. 2025Visual Attention Never Fades: Selective Progressive Attention ReCalibration for Detailed Image Captioning in Multimodal Large Language Modelscited by 22
  18. 2024Mitigating Hallucination in Multimodal Large Language Model via Hallucination-targeted Direct Preference Optimizationcited by 20
  19. 2024Toward Robust Hyper-Detailed Image Captioning: A Multiagent Approach and Dual Evaluation Metrics for Factuality and Coveragecited by 20
  20. 2025Iterative Tool Usage Exploration for Multimodal Agents via Step-wise Preference Tuningcited by 19
  21. 2024EFUF: Efficient Fine-grained Unlearning Framework for Mitigating Hallucinations in Multimodal Large Language Modelscited by 17
  22. 2024Pelican: Correcting Hallucination in Vision-LLMs via Claim Decomposition and Program of Thought Verificationcited by 17
  23. 2024Sycophancy in Vision-Language Models: A Systematic Analysis and an Inference-Time Mitigation Frameworkcited by 17
  24. 2025PaMi-VDPO: Mitigating Video Hallucinations by Prompt-Aware Multi-Instance Video Preference Learningcited by 17
  25. 2024SciDoc2Diagrammer-MAF: Towards Generation of Scientific Diagrams from Documents guided by Multi-Aspect Feedback Refinementcited by 15
  26. 2025Mirage in the Eyes: Hallucination Attack on Multi-modal Large Language Models with Only Attention Sinkcited by 15
  27. 2025Hallucinatory Image Tokens: A Training-free EAZY Approach on Detecting and Mitigating Object Hallucinations in LVLMscited by 15
  28. 2025Learning to Steer: Input-dependent Steering for Multimodal LLMscited by 15
  29. 2025RefineBench: Evaluating Refinement Capability of Language Models via Checklistscited by 15
  30. 2024Pensieve: Retrospect-then-Compare Mitigates Visual Hallucinationcited by 14
  31. 2025Perception in Reflectioncited by 13
  32. 2025Visual Multi-Agent System: Mitigating Hallucination Snowballing via Visual Flowcited by 13
  33. 2025Skip-Vision: Efficient and Scalable Acceleration of Vision-Language Models via Adaptive Token Skippingcited by 12
  34. 2025Do You Keep an Eye on What I Ask? Mitigating Multimodal Hallucination via Attention-Guided Ensemble Decodingcited by 12
  35. 2024Visual Hallucination: Definition, Quantification, and Prescriptive Remediationscited by 11
  36. 2025MedHEval: Benchmarking Hallucinations and Mitigation Strategies in Medical Large Vision-Language Modelscited by 10
  37. 2025Antidote: A Unified Framework for Mitigating LVLM Hallucinations in Counterfactual Presupposition and Object Perceptioncited by 10
  38. 2025Mitigating Hallucinations in Large Vision-Language Models via Entity-Centric Multimodal Preference Optimizationcited by 10

50 most cited of 98 All 98 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Self-Correcting Decoding with Generative Feedback for Mitigating Hallucinations in Large Vision-Language Models
  2. 2025Generate, but Verify: Reducing Hallucination in Vision-Language Models with Retrospective Resampling
  3. 2024Hallucination in Multimodal Foundation Models: A Survey on Causes, Corrections, and Evaluations
  4. 2026Kestrel: Grounding Self-Refinement for LVLM Hallucination Mitigation
  5. 2025SAVER: Mitigating Hallucinations in Large Vision-Language Models via Style-Aware Visual Early Revision
  6. 2026Mitigating Multimodal Hallucination via Phase-wise Self-reward
  7. 2024Less is More: Mitigating Multimodal Hallucination from an EOS Decision Perspective
  8. 2026Self-Correction Inside the Model: Leveraging Layer Attention to Mitigate Hallucinations in Large Vision Language Models