mlsanityData to 14 Oct 2026

Interpreting Attention Heads for Image-to-Text Information Flow in Large Vision-Language Models

arXiv 2509.17588 ↗First posted 22 Sep 2025cs.CV cs.AI cs.LGcited by 9

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Large Vision-Language Models (LVLMs) answer visual questions by transferring information from images to text through a series of attention heads. While this image-to-text information flow is central to visual question answering, its underlying mechanism remains difficult to interpret due to the simultaneous operation of numerous attention heads. To address this challenge, we propose head attribution, a technique inspired by component attribution methods, to identify consistent patterns among attention heads that play a key role in information transfer. Using head attribution, we investigate how LVLMs rely on specific attention heads to identify and answer questions about the main object in an image. Our analysis reveals that a distinct subset of attention heads facilitates the image-to-text information flow. Remarkably, we find that the selection of these heads is governed by the semantic content of the input image rather than its visual appearance. We further examine the flow of information at the token level and discover that (1) text information first propagates to role-related tokens and the final token before receiving image information, and (2) image information is embedded in both object-related and background tokens. Our work provides evidence that image-to-text information flow follows a structured process, and that analysis at the attention-head level offers a promising direction toward understanding the mechanisms of LVLMs.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 52 of its 65 references that are in the map. First posted on arXiv 22 Sep 2025.

Builds on

52 of 65 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024What's in the Image? A Deep-Dive into the Vision of Vision Language Models26 Nov
  2. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?29 Mar

Other references · 50

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2016Layer Normalizationcited by 5,669
  5. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  6. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  7. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  8. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  9. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
  10. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  11. 2019Root Mean Square Layer Normalizationcited by 1,480
  12. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,331
38 more
  1. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  2. 2022Toy Models of Superpositioncited by 971
  3. 2019Attention is not Explanationcited by 929
  4. 2023Vision Transformers Need Registerscited by 871
  5. 2018Object Hallucination in Image Captioningcited by 669
  6. 2024An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Modelscited by 642
  7. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 528
  8. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
  9. 2021Causal Abstractions of Neural Networkscited by 422
  10. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
  11. 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inferencecited by 391
  12. 2024VisionZip: Longer is Better but Not Necessary in Vision Language Modelscited by 313
  13. 2024PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reductioncited by 294
  14. 2023Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learningcited by 236
  15. 2024How to use and interpret activation patchingcited by 214
  16. 2023How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language modelcited by 202
  17. 2023The Hydra Effect: Emergent Self-repair in Language Model Computationscited by 156
  18. 2023Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchillacited by 149
  19. 2024The Remarkable Robustness of LLMs: Stages of Inference?cited by 146
  20. 2024Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lenscited by 143
  21. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 129
  22. 2024Fit and Prune: Fast and Training-free Visual Token Pruning for Multi-modal Large Language Modelscited by 126
  23. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 118
  24. 2024AtP*: An efficient and scalable method for localizing LLM behaviour to componentscited by 111
  25. 2024Interpreting and Editing Vision-Language Representations to Mitigate Hallucinationscited by 104
  26. 2024Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Modelscited by 91
  27. 2024What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formationcited by 89
  28. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 85
  29. 2024What Matters in Transformers? Not All Attention is Neededcited by 84
  30. 2024Attention Heads of Large Language Models: A Surveycited by 67
  31. 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Surveycited by 52
  32. 2024ZipVL: Efficient Large Vision-Language Models with Dynamic Token Sparsificationcited by 44
  33. 2024Explorations of Self-Repair in Language Modelscited by 39
  34. 2024Optimal ablation for interpretabilitycited by 25
  35. 2023A Glitch in the Matrix? Locating and Detecting Language Model Grounding with Fakepediacited by 24
  36. 2024Decomposing and Editing Predictions by Modeling Model Computationcited by 19
  37. 2024Accelerating Multimodal Large Language Models via Dynamic Visual-Token Exit and the Empirical Findingscited by 15
  38. 2025Position-aware Automatic Circuit Discoverycited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 8.

9 papers.

  1. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 27
  2. 2026Pathways of Visual Information Flow in Vision-Language Modelscited by 5
  3. 2026When Sinks Help or Hurt: Unified Framework for Attention Sink in Large Vision-Language Modelscited by 4
  4. 2025Multimodal LLMs for Historical Dataset Construction from Archival Image Scans: German Patents (1877-1918)cited by 2
  5. 2026Interpretable Motion-Attentive Maps: Spatio-Temporally Localizing Concepts in Video Diffusion Transformerscited by 1
  6. 2026MLLMs Know When Before Speaking: Revealing and Recovering Temporal Grounding via Attention Cuescited by 1
  7. 2026Ego: Embedding-Guided Personalization of Vision-Language Modelscited by 0
  8. 2026What Do Visual-Head Discovery Scores Actually Find? Attention Source, Necessity, and Random-Control Pitfalls in Vision-Language Modelscited by 0
  9. 2026Understanding Unified VLMs: From Attention Probing to Guidance Tuning for Multimodal Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Aligning What Vision-Language Models See and Perceive with Adaptive Information Flow
  2. 2024Cross-modal Information Flow in Multimodal Large Language Models
  3. 2026Retrieval Heads Meet Vision: Uncovering How VLMs Locate and Extract Visual Information
  4. 2024From Redundancy to Relevance: Information Flow in LVLMs Across Reasoning Tasks
  5. 2026Do Vision-Language Models See or Guess? Measuring and Reducing Textual-Prior Reliance with a Phrasing-Controlled Benchmark
  6. 2025Head Pursuit: Probing Attention Specialization in Multimodal Transformers
  7. 2025Direct Visual Grounding by Directing Attention of Visual Tokens
  8. 2026Can Retrieval Heads See Images? Multimodal Retrieval Heads in Long-Context Vision-Language Models