mlsanityData to 14 Oct 2026

See Different, Think Better: Visual Variations Mitigating Hallucinations in LVLMs

arXiv 2507.22003 ↗First posted 29 Jul 2025cs.CVcited by 4

On the map: LLM reasoning → Hallucination detection and mitigation

also in 1 more
Abstract (arXiv)

Large Vision-Language Models (LVLMs) have demonstrated remarkable capabilities in visual understanding and multimodal reasoning. However, LVLMs frequently exhibit hallucination phenomena, manifesting as the generated textual responses that demonstrate inconsistencies with the provided visual content. Existing hallucination mitigation methods are predominantly text-centric, the challenges of visual-semantic alignment significantly limit their effectiveness, especially when confronted with fine-grained visual understanding scenarios. To this end, this paper presents ViHallu, a Vision-Centric Hallucination mitigation framework that enhances visual-semantic alignment through Visual Variation Image Generation and Visual Instruction Construction. ViHallu introduces visual variation images with controllable visual alterations while maintaining the overall image structure. These images, combined with carefully constructed visual instructions, enable LVLMs to better understand fine-grained visual content through fine-tuning, allowing models to more precisely capture the correspondence between visual content and text, thereby enhancing visual-semantic alignment. Extensive experiments on multiple benchmarks show that ViHallu effectively enhances models' fine-grained visual understanding while significantly reducing hallucination tendencies. Furthermore, we release ViHallu-Instruction, a visual instruction dataset specifically designed for hallucination mitigation and visual-semantic alignment. Code is available at https://github.com/oliviadzy/ViHallu.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Hallucination detection and mitigation, one of 17 directions in LLM reasoning. Builds on 21 of its 41 references that are in the map. First posted on arXiv 29 Jul 2025.

Builds on

21 of 41 references.

References · 21

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  4. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  5. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  6. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,351
  7. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  8. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  9. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 712
  10. 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 474
  11. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Modelscited by 316
  12. 2023Composer: Creative and Controllable Image Synthesis with Composable Conditionscited by 304
9 more
  1. 2024mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Modelscited by 291
  2. 2023Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimizationcited by 225
  3. 2020Deep Structural Causal Models for Tractable Counterfactual Inferencecited by 211
  4. 2024ControlNet++: Improving Conditional Controls with Efficient Consistency Feedbackcited by 154
  5. 2023To See is to Believe: Prompting GPT-4V for Better Visual Instruction Tuningcited by 144
  6. 2020Boosting Contrastive Self-Supervised Learning with False Negative Cancellationcited by 119
  7. 2023Mitigating Fine-Grained Hallucination by Fine-Tuning Large Vision-Language Models with Caption Rewritescited by 61
  8. 2024Alleviating Hallucination in Large Vision-Language Models with Active Retrieval Augmentationcited by 29
  9. 2023Enhancing Multimodal Compositional Reasoning of Visual Language Models with Generative Negative Miningcited by 25

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2026MTAVG-Bench 2.0: Diagnosing Failure Modes of Cinematic Expressiveness in Multi-Talker Audio-Video Generationcited by 4
  2. 2026When Does Visual Generation Help Visual Understanding in Unified Multimodal Models?cited by 2
  3. 2025MMRPT: MultiModal Reinforcement Pre-Training via Masked Vision-Dependent Reasoningcited by 1
  4. 2026MaskSense: Confronting the Visual Exploration Trap in Masked Image Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026What Makes LVLMs Hallucinate Less? Unveiling the Architectural Factors Behind Hallucination Robustness
  2. 2024Reducing Hallucinations in Vision-Language Models via Latent Space Steering
  3. 2024Cracking the Code of Hallucination in LVLMs with Vision-aware Head Divergence
  4. 2025SAVER: Mitigating Hallucinations in Large Vision-Language Models via Style-Aware Visual Early Revision
  5. 2024From Pixels to Tokens: Revisiting Object Hallucinations in Large Vision-Language Models
  6. 2025Intervene-All-Paths: Unified Mitigation of LVLM Hallucinations across Alignment Formats
  7. 2026Reducing Object Hallucination in LVLMs via Emphasizing Image-negative Tokens
  8. 2025A Comprehensive Analysis for Visual Object Hallucination in Large Vision-Language Models