mlsanityData to 14 Oct 2026

Look Carefully: Adaptive Visual Reinforcements in Multimodal Large Language Models for Hallucination Mitigation

arXiv 2602.24041 ↗First posted 27 Feb 2026cs.CVcited by 7

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Multimodal large language models (MLLMs) have achieved remarkable progress in vision-language reasoning, yet they remain vulnerable to hallucination, where generated content deviates from visual evidence. Existing mitigation strategies either require costly supervision during training or introduce additional latency at inference time. Recent vision enhancement methods attempt to address this issue by reinforcing visual tokens during decoding, but they typically inject all tokens indiscriminately, which causes interference from background regions and distracts the model from critical cues. To overcome this challenge, we propose Adaptive Visual Reinforcement (AIR), a training-free framework for MLLMs. AIR consists of two components. Prototype-based token reduction condenses the large pool of visual tokens into a compact subset to suppress redundancy. OT-guided patch reinforcement quantifies the alignment between hidden states and patch embeddings to selectively integrate the most consistent patches into feed-forward layers. As a result, AIR enhances the model's reliance on salient visual information and effectively mitigates hallucination. Extensive experiments across representative MLLMs demonstrate that AIR substantially reduces hallucination while preserving general capabilities, establishing it as an effective solution for building reliable MLLMs.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 53 of its 62 references that are in the map. First posted on arXiv 27 Feb 2026.

Builds on

53 of 62 references.

Closest ancestors · 19

Same direction, newest first

  1. 2025Modality Bias in LVLMs: Analyzing and Mitigating Object Hallucination via Attention Lens4 Aug
  2. 2025Mitigating Object Hallucinations via Sentence-Level Early Intervention16 Jul
  3. 2025ONLY: One-Layer Intervention Sufficiently Mitigates Hallucinations in Large Vision-Language Models1 Jul
  4. 2025Mitigating Hallucination in Large Vision-Language Models via Adaptive Attention Calibration27 May
  5. 2025ClearSight: Visual Signal Enhancement for Object Hallucination Mitigation in Multimodal Large language Models17 Mar
  6. 2025Mitigating Hallucinations in Large Vision-Language Models with Internal Fact-based Contrastive Decoding3 Feb
  7. 2025CHiP: Cross-modal Hierarchical Direct Preference Optimization for Multimodal LLMs28 Jan
  8. 2025Mitigating Hallucinations in Large Vision-Language Models via DPO: On-Policy Data Hold the Key16 Jan
  9. 2024Nullu: Mitigating Object Hallucinations in Large Vision-Language Models via HalluSpace Projection18 Dec
  10. 2024Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens23 Nov
  11. 2024Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Models4 Oct
  12. 2024Reefknot: A Comprehensive Benchmark for Relation Hallucination Evaluation, Analysis and Mitigation in Multimodal Large Language Models18 Aug
7 more
  1. 2024Alleviating Hallucinations in Large Vision-Language Models through Hallucination-Induced Optimization24 May
  2. 2024Logical Closed Loop: Uncovering Object Hallucinations in Large Vision-Language Models18 Feb
  3. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
  4. 2023Woodpecker: Hallucination Correction for Multimodal Large Language Models24 Oct
  5. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Models1 Oct
  6. 2023Detecting and Preventing Hallucinations in Large Vision Language Models11 Aug
  7. 2023Evaluating Object Hallucination in Large Vision-Language Models17 May

Other references · 34

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2023Visual Instruction Tuningcited by 7,869
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  4. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  5. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  6. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,316
  7. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  8. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  9. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  10. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 710
  11. 2018Object Hallucination in Image Captioningcited by 669
  12. 2025On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectificationcited by 127
22 more
  1. 2025KRIS-Bench: Benchmarking Next-Level Intelligent Image Editing Modelscited by 71
  2. 2024Unlearning Concepts in Diffusion Model via Concept Domain Correction and Concept Preserving Gradientcited by 69
  3. 2024Number it: Temporal Grounding Videos like Flipping Mangacited by 65
  4. 2023Exploring Diverse In-Context Configurations for Image Captioningcited by 63
  5. 2022Unified Optimal Transport Framework for Universal Domain Adaptationcited by 57
  6. 2020Iterative label cleaning for transductive and semi-supervised few-shot learningcited by 40
  7. 2020Multi-source Domain Adaptation via Weighted Joint Distributions Optimal Transportcited by 28
  8. 2024AWT: Transferring Vision-Language Models via Augmentation, Weighting, and Transportationcited by 28
  9. 2023Tuning Multi-mode Token-level Prompt Alignment across Modalitiescited by 26
  10. 2024Enhancing Zero-Shot Vision Models by Label-Free Prompt Distribution Learning and Bias Correctingcited by 26
  11. 2025Unsupervised Visual Chain-of-Thought Reasoning via Preference Optimizationcited by 22
  12. 2023PatchCT: Aligning Patch Set and Label Set with Conditional Transport for Multi-Label Image Classificationcited by 21
  13. 2024Selective Vision-Language Subspace Projection for Few-shot CLIPcited by 18
  14. 2025Enhancing CLIP Robustness via Cross-Modality Alignmentcited by 15
  15. 2022Adaptive Distribution Calibration for Few-Shot Learning with Hierarchical Optimal Transportcited by 14
  16. 2025Dynamic Multimodal Prototype Learning in Vision-Language Modelscited by 13
  17. 2025Benchmarking and Bridging Emotion Conflicts for Multimodal Emotion Reasoningcited by 12
  18. 2024Boosting Few-Shot Learning via Attentive Feature Regularizationcited by 10
  19. 2024Video Repurposing from User Generated Content: A Large-scale Dataset and Benchmarkcited by 9
  20. 2024CARE Transformer: Mobile-Friendly Linear Visual Transformer via Decoupled Dual Interactioncited by 5
  21. 2025Hierarchical Semantic Alignment for Image Clusteringcited by 5
  22. 2021Few-shot Partial Multi-view Learningcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 7.

7 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2024Cracking the Code of Hallucination in LVLMs with Vision-aware Head Divergence
  2. 2024VaLiD: Mitigating the Hallucination of Large Vision Language Models by Visual Layer Fusion Contrastive Decoding
  3. 2026Beyond Visual Enhancement: Adaptive Multi-Context Steering to Mitigate LVLM Hallucinations
  4. 2026ACT Now: Preempting LVLM Hallucinations via Adaptive Context Integration
  5. 2026Mitigating Hallucination in Vision-Language Models through Barrier-Regulated Adaptive Closed-form Steering
  6. 2026Hallucination-aware intermediate representation edit in large vision-language models
  7. 2025Not All Tokens and Heads Are Equally Important: Dual-Level Attention Intervention for Hallucination Mitigation
  8. 2025Mitigating Image Captioning Hallucinations in Vision-Language Models