mlsanityData to 7 Oct 2026

GLIMPSE: Holistic Cross-Modal Explainability for Large Vision-Language Models

arXiv 2506.18985 ↗First posted 23 Jun 2025cs.CV cs.AIcited by 5

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Recent large vision-language models (LVLMs) have advanced capabilities in visual question answering (VQA). However, interpreting where LVLMs direct their visual attention remains a significant challenge, yet is essential for understanding model behavior. We introduce GLIMPSE (Gradient-Layer Importance Mapping for Prompted Visual Saliency Explanation), a lightweight, model-agnostic framework that jointly attributes LVLM outputs to the most relevant visual evidence and textual signals that support open-ended generation. GLIMPSE fuses gradient-weighted attention, adaptive layer propagation, and relevance-weighted token aggregation to produce holistic response-level heat maps for interpreting cross-modal reasoning, outperforming prior methods in faithfulness and pushing the state-of-the-art in human-attention alignment. We demonstrate an analytic approach to uncover fine-grained insights into LVLM cross-modal attribution, trace reasoning dynamics, analyze systematic misalignment, diagnose hallucination and bias, and ensure transparency.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 9 of its 28 references that are in the map. First posted on arXiv 23 Jun 2025.

Builds on

9 of 28 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Attention, Please! PixelSHAP Reveals What Vision-Language Models Actually Focus On9 Mar
  2. 2024LVLM-Interpret: An Interpretability Tool for Large Vision-Language Models3 Apr
  3. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond24 Aug

Other references · 6

  1. 2023Visual Instruction Tuningcited by 7,792
  2. 2025Qwen2.5-VL Technical Reportcited by 5,418
  3. 2024AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformerscited by 116
  4. 2023AtMan: Understanding Transformer Predictions Through Memory Efficient Attention Manipulationcited by 30
  5. 2023Visual Explanations via Iterated Integrated Attributionscited by 18
  6. 2024Uncertainty estimates for semantic segmentation: providing enhanced reliability for automated motor claims handlingcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026Saliency-R1: Enforcing Interpretable and Faithful Vision-language Reasoning via Saliency-map Alignment Rewardcited by 1
  2. 2026Through Their Eyes: Fixation-aligned Tuning for Personalized User Emulationcited by 0
  3. 2026Real-Time Visual Attribution Streaming in Thinking Modelcited by 0
  4. 2026AnswerMap: Faithful Spatial Interpretability of VLMs from Answer Posteriorscited by 0
  5. 2026Tracing the Evidence: Faithful Token Attribution Through Vision-Language Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Measuring Cross-Modal Synergy: A Benchmark for VLM Explainability
  2. 2025Cross-Modal Attention Guided Unlearning in Vision-Language Models
  3. 2025Where do Large Vision-Language Models Look at when Answering Questions?
  4. 2024Unraveling Cross-Modality Knowledge Conflicts in Large Vision-Language Models
  5. 2025Exploring Implicit Visual Misunderstandings in Multimodal Large Language Models through Attention Analysis
  6. 2026Where Does Vision Meet Language? Understanding and Refining Visual Fusion in MLLMs via Contrastive Attention
  7. 2024FiVL: A Framework for Improved Vision-Language Alignment through the Lens of Training, Evaluation and Explainability
  8. 2024Cross-modal Information Flow in Multimodal Large Language Models