mlsanityData to 14 Oct 2026

Head Pursuit: Probing Attention Specialization in Multimodal Transformers

arXiv 2510.21518 ↗First posted 24 Oct 2025cs.CV cs.CL cs.LGcited by 17

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Language and vision-language models have shown impressive performance across a wide range of tasks, but their internal mechanisms remain only partly understood. In this work, we study how individual attention heads in text-generative models specialize in specific semantic or visual attributes. Building on an established interpretability method, we reinterpret the practice of probing intermediate activations with the final decoding layer through the lens of signal processing. This lets us analyze multiple samples in a principled way and rank attention heads based on their relevance to target concepts. Our results show consistent patterns of specialization at the head level across both unimodal and multimodal transformers. Remarkably, we find that editing as few as 1% of the heads, selected using our method, can reliably suppress or enhance targeted concepts in the model output. We validate our approach on language tasks such as question answering and toxicity mitigation, as well as vision-language tasks including image classification and captioning. Our findings highlight an interpretable and controllable structure within attention layers, offering simple tools for understanding and editing large-scale generative models.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 36 of its 51 references that are in the map. First posted on arXiv 24 Oct 2025.

Builds on

36 of 51 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Analyzing Finetuning Representation Shift for Multimodal LLMs Steering6 Jan
  2. 2024A Concept-Based Explainability Framework for Large Multimodal Models12 Jun

Other references · 34

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023Visual Instruction Tuningcited by 7,869
  4. 2025Qwen2.5-VL Technical Reportcited by 5,587
  5. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  6. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  7. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  8. 2013Describing Textures in the Wildcited by 2,125
  9. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,046
  10. 2025Gemma 3 Technical Reportcited by 1,824
  11. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,496
  12. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
22 more
  1. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 856
  2. 2019Are Sixteen Heads Really Better than One?cited by 816
  3. 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 797
  4. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
  5. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
  6. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 528
  7. 2023Interpreting CLIP's Image Representation via Text-Based Decompositioncited by 196
  8. 2022CRAFT: Concept Recursive Activation FacTorization for Explainabilitycited by 178
  9. 2022Analyzing Transformers in Embedding Spacecited by 132
  10. 2024Towards Interpreting Visual Information Processing in Vision-Language Modelscited by 122
  11. 2024On the Origins of Linear Representations in Large Language Modelscited by 79
  12. 2024Understanding Information Storage and Transfer in Multi-modal Large Language Modelscited by 49
  13. 2024Talking Heads: Understanding Inter-layer Communication in Transformer Language Modelscited by 41
  14. 2024Competition of Mechanisms: Tracing How Language Models Handle Facts and Counterfactualscited by 40
  15. 2025Interpreting the linear structure of vision-language model embedding spacescited by 32
  16. 2023Attention Lens: A Tool for Mechanistically Interpreting the Attention Head Information Retrieval Mechanismcited by 22
  17. 2024Realistic Evaluation of Toxicity in Large Language Modelscited by 15
  18. 2025When Seeing Overrides Knowing: Disentangling Knowledge Conflicts in Vision-Language Modelscited by 11
  19. 2024ResiDual Transformer Alignment with Spectral Decompositioncited by 6
  20. 2025Line of Sight: On Linear Representations in VLLMscited by 6
  21. 2023Decoding Layer Saliency in Language Transformerscited by 5
  22. 2024The Narrow Gate: Localized Image-Text Communication in Native Multimodal Modelscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 17.

17 papers.

Sort:

By area:

  1. 2026Large Vision-Language Models Get Lost in Attentioncited by 5
  2. 2026Towards Understanding Modality Interaction in Multimodal Language Models via Partial Information Decompositioncited by 5
  3. 2026Are Audio-Language Models Listening? Audio-Specialist Heads for Adaptive Audio Steeringcited by 4
  4. 2026Mechanistic Insights into Functional Sparsity in Multimodal LLMs via CoRe Headscited by 4
  5. 2026Multimodal In-context Learning for ASR of Low-resource Languagescited by 2
  6. 2026Interpreting Transformers Through Attention Head Interventioncited by 1
  7. 2026Softmax Linear Attention: Reclaiming Global Competitioncited by 1
  8. 2026Embedding-Aware Feature Discovery: Bridging Latent Representations and Interpretable Features in Event Sequencescited by 1
  9. 2026Knowing Isn't Always Saying: When Do Spatial Encodings Reach Answers in Vision-Language Models?cited by 1
  10. 2026Semantic Head Specialization Guides Hybrid ViT Attention for Multimodal LLMscited by 1
  11. 2026CALM: Class-Conditional Sparse Attention Vectors for Large Audio-Language Modelscited by 0
  12. 2026Do Foundation Models Know Geometry? Probing Frozen Features for Continuous Physical Measurementcited by 0
5 more
  1. 2026HaloProbe: Bayesian Detection and Mitigation of Object Hallucinations in Vision-Language Modelscited by 0
  2. 2026Mitigating Action-Relation Hallucinations in LVLMs via Relation-aware Visual Enhancementcited by 0
  3. 2026LocUS: Head Selection and Subspace Projection for Targeted Activation Steeringcited by 0
  4. 2026Look Before You Judge: Training-Free Region Mining for Grounded and Explainable Deepfake Detectioncited by 0
  5. 2026Emergent Structure in the Marginal Attention Space of Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Unveiling Visual Perception in Language Models: An Attention Head Analysis Approach
  2. 2025Investigating The Functional Roles of Attention Heads in Vision Language Models: Evidence for Reasoning Modules
  3. 2026Attending to Multimodal Generation One Token at a Time
  4. 2025How Multimodal LLMs Solve Image Tasks: A Lens on Visual Grounding, Task Reasoning, and Answer Decoding
  5. 2026Where Does Vision Meet Language? Understanding and Refining Visual Fusion in MLLMs via Contrastive Attention
  6. 2025Perceiving Beyond Language Priors: Enhancing Visual Comprehension and Attention in Multimodal Models
  7. 2025Interpreting Attention Heads for Image-to-Text Information Flow in Large Vision-Language Models
  8. 2026Mind What Matters for Reasoning: Aligning Cross-Modal Attention via Selective Probability Mass Concentration