mlsanityData to 7 Oct 2026

DICA: Dual-Indicator Guided Contrastive Alignment in Multimodal Large Language Models

arXiv 2607.23944 ↗First posted 27 Jul 2026cs.AIcited by 0

On the map: LLM reasoning → Hallucination detection and mitigation

also in 1 more
Abstract (arXiv)

Human visual reasoning typically follows a coarse-to-fine attention process, starting from global scene understanding and gradually focusing on question-relevant regions. However, multimodal large language models may deviate from this pattern due to attention drift and the underutilization of visual evidence, which can lead to hallucinations. To mitigate these issues, this study proposes a Dual-Indicator Guided Contrastive Alignment (DICA), which tracks two information-theoretic indicators during inference: Visual Attention Entropy (VAE), which reflects the concentration of visual attention, and Output Image Correlation (OIC), which measures the dependence of generated outputs on the visual input. An abnormal increase in VAE or a decrease in OIC corresponds to different failure modes, which trigger targeted contrastive alignment to restore visual grounding. Experimental results across multiple benchmarks demonstrate that DICA consistently outperforms existing approaches and substantially reduces hallucinations, highlighting the effectiveness of indicator-driven intervention in improving multimodal inference reliability. The code is publicly available at https://github.com/BGWH123/DICA/.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Hallucination detection and mitigation, one of 17 directions in LLM reasoning. Builds on 24 of its 44 references that are in the map. First posted on arXiv 27 Jul 2026.

Builds on

24 of 44 references.

References · 24

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  3. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  4. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  5. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,075
  6. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
  7. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 702
  8. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 671
  9. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 598
  10. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocationcited by 468
  11. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuningcited by 439
  12. 2024Multi-Modal Hallucination Control by Visual Information Groundingcited by 196
12 more
  1. 2023LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examplescited by 177
  2. 2023ViP-LLaVA: Making Large Multimodal Models Understand Arbitrary Visual Promptscited by 160
  3. 2016Human Attention in Visual Question Answering: Do Humans and Deep Networks Look at the Same Regions?cited by 111
  4. 2021Let there be a clock on the beach: Reducing Object Hallucination in Image Captioningcited by 74
  5. 2023Mitigating Fine-Grained Hallucination by Fine-Tuning Large Vision-Language Models with Caption Rewritescited by 61
  6. 2023Instructive Decoding: Instruction-Tuned Large Language Models are Self-Refiner from Noisy Instructionscited by 24
  7. 2024Enhancing Semantics in Multimodal Chain of Thought via Soft Negative Samplingcited by 8
  8. 2025EAGLE: Enhanced Visual Grounding Minimizes Hallucinations in Instructional Multimodal Modelscited by 8
  9. 2025Collaborative Multi-LoRA Experts with Achievement-based Multi-Tasks Loss for Unified Multimodal Information Extractioncited by 7
  10. 2025Vision-aware Multimodal Prompt Tuning for Uploadable Multi-source Few-shot Domain Adaptationcited by 3
  11. 2025Hybrid-DMKG: A Hybrid Reasoning Framework over Dynamic Multimodal Knowledge Graphs for Multimodal Multihop QA with Knowledge Editingcited by 2
  12. 2026EVL-MCoT: Enhanced Vision-Language Multi-CoT for Harmful Meme Detectioncited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Language Priors: Diagnosing and Fixing Visual-Origin Hallucinations in Multimodal LLM
  2. 2024Mitigating Modality Prior-Induced Hallucinations in Multimodal Large Language Models via Deciphering Attention Causality
  3. 2025Mitigating Object Hallucinations in Large Vision-Language Models via Attention Calibration
  4. 2025Causal-LLaVA: Causal Disentanglement for Mitigating Hallucination in Multimodal Large Language Models
  5. 2026Mitigating Visual Hallucinations in Multimodal Systems through Retrieval-Augmented Reliability-Aware Inference
  6. 2025Causally-Grounded Dual-Path Attention Intervention for Object Hallucination Mitigation in LVLMs
  7. 2025V-ITI: Mitigating Hallucinations in Multimodal Large Language Models via Visual Inference-Time Intervention
  8. 2025Mitigating Hallucination in Large Vision-Language Models via Adaptive Attention Calibration