mlsanityData to 14 Oct 2026

LVLM-Interpret: An Interpretability Tool for Large Vision-Language Models

arXiv 2404.03118 ↗First posted 3 Apr 2024cs.CVcited by 24

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

also in 1 more
Abstract (arXiv)

In the rapidly evolving landscape of artificial intelligence, multi-modal large language models are emerging as a significant area of interest. These models, which combine various forms of data input, are becoming increasingly popular. However, understanding their internal mechanisms remains a complex task. Numerous advancements have been made in the field of explainability tools and mechanisms, yet there is still much to explore. In this work, we present a novel interactive application aimed towards understanding the internal mechanisms of large vision-language models. Our interface is designed to enhance the interpretability of the image patches, which are instrumental in generating an answer, and assess the efficacy of the language model in grounding its output in the image. With our application, a user can systematically investigate the model and uncover system limitations, paving the way for enhancements in system capabilities. Finally, we present a case study of how our application can aid in understanding failure mechanisms in a popular large multi-modal model: LLaVA.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 11 of its 50 references that are in the map. First posted on arXiv 3 Apr 2024.

Builds on

11 of 50 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond24 Aug

Other references · 10

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 872
  4. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 735
  5. 2024Hallucination is Inevitable: An Innate Limitation of Large Language Modelscited by 332
  6. 2023Evaluation and Analysis of Hallucination in Large Vision-Language Modelscited by 99
  7. 2022ProtoPFormer: Concentrating on Prototypical Parts in Vision Transformers for Interpretable Image Recognitioncited by 63
  8. 2023Interpretability-Aware Vision Transformercited by 8
  9. 2022CLEAR: Causal Explanations from Attention in Neural Recommenderscited by 4
  10. 2024ComFe: An Interpretable Head for Vision Transformerscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 12; 2026: 6.

24 papers.

Sort:

By area:

  1. 2025See What You Are Told: Visual Attention Sink in Large Multimodal Modelscited by 132
  2. 2024DeCo: Decoupling Token Compression from Semantic Abstraction in Multimodal Large Language Modelscited by 75
  3. 2024ControlMLLM: Training-Free Visual Prompt Learning for Multimodal Large Language Modelscited by 33
  4. 2024Survey of Large Multimodal Model Datasets, Application Categories and Taxonomycited by 17
  5. 2025MVI-Bench: A Comprehensive Benchmark for Evaluating Robustness to Misleading Visual Inputs in LVLMscited by 15
  6. 2025Backdoor Attack on Vision Language Models with Stealthy Semantic Manipulationcited by 14
  7. 2025Probing the Representational Power of Sparse Autoencoders in Vision Modelscited by 10
  8. 2025V-SEAM: Visual Semantic Editing and Attention Modulating for Causal Interpretability of Vision-Language Modelscited by 8
  9. 2026How Vision Becomes Language: A Layer-wise Information-Theoretic Analysis of Multimodal Reasoningcited by 8
  10. 2025SurgXBench: Explainable Vision-Language Model Benchmark for Surgerycited by 6
  11. 2025GLIMPSE: Holistic Cross-Modal Explainability for Large Vision-Language Modelscited by 5
  12. 2025Evidence Recomposition and Predictive Context Residualization for Visual Attribution in Multimodal Large Language Modelscited by 5
12 more
  1. 2025Can VLMs Detect and Localize Fine-Grained AI-Edited Images?cited by 4
  2. 2024FastRM: An efficient and automatic explainability framework for multimodal generative modelscited by 2
  3. 2025On the Risk of Misleading Reports: Diagnosing Textual Biases in Multimodal Clinical AIcited by 2
  4. 2024VISTA: A Visual and Textual Attention Dataset for Interpreting Multimodal Modelscited by 1
  5. 2025Augmented Vision-Language Models: A Systematic Reviewcited by 1
  6. 2026Towards Understanding Multimodal Fine-Tuning: Spatial Featurescited by 1
  7. 2024Advancing Histopathology-Based Breast Cancer Diagnosis: Insights into Multi-Modality and Explainabilitycited by 0
  8. 2025Adapting Lightweight Vision Language Models for Radiological Visual Question Answeringcited by 0
  9. 2026Structural Graph Probing of Vision-Language Modelscited by 0
  10. 2026Real-Time Visual Attribution Streaming in Thinking Modelcited by 0
  11. 2026Multimodal Model Diffing for Feature Discovery and Controlcited by 0
  12. 2026When, Where, and Why: Hierarchical Causal Explainability for Video Face Understanding with MLLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey
  2. 2023ReForm-Eval: Evaluating Large Vision Language Models via Unified Re-Formulation of Task-Oriented Benchmarks
  3. 20233VL: Using Trees to Improve Vision-Language Models' Interpretability
  4. 2024LM4LV: A Frozen Large Language Model for Low-level Vision Tasks
  5. 2024Explaining Multi-modal Large Language Models by Analyzing their Vision Perception
  6. 2026More Images, More Problems? A Controlled Analysis of VLM Failure Modes
  7. 2024An Introduction to Vision-Language Modeling
  8. 2024A Comprehensive Survey and Guide to Multimodal Large Language Models in Vision-Language Tasks