mlsanityData to 14 Oct 2026

What Do Visual-Head Discovery Scores Actually Find? Attention Source, Necessity, and Random-Control Pitfalls in Vision-Language Models

arXiv 2610.13111 ↗First posted 9 Oct 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Attention heads that attend to image regions relevant to a question are increasingly used to steer, ground and interpret vision-language models (VLMs) and to reduce their hallucinations. They are mostly found by ranking heads with an attention- pattern score and validated by steering or masking them. We ask how the choices inside such a discovery score shape the heads it returns, and which of the two validations identifies them. On Qwen3-VL-8B-Instruct we vary the query type, the discovery corpus and the token positions whose attention is read (the attention source) in a full 2 x 3 x 2 design, and test all 12 head-sets for steerability (CERS: Contrastive Evidence-Region Steering score) and necessity (NCS: Necessity Con- trast Score) against layer-matched random head-sets. (1) The attention source and the query type reshape the selected heads far more than the corpus: changing the source keeps a median of 49 of the top-100 heads, the query 55 and the cor- pus 84, while two halves of the same data share 96-99. (2) Necessity follows the attention source: head-sets discovered from query-span attention are more nec- essary than last-token ones (difference 0.083, p=6 x 10^-10), and the two re- ferring + query-span sets lie 17-19 standard deviations above 250 layer-matched random sets (Holm-adjusted p=0.048). The referring + query-span effect re- plicates on several recent VLMs. (3) Steerability is not specific to the discovered heads: no discovered head-set steers better than its layer-matched random sets (rank p >= 0.14), and random sets drawn from the same or from any layer span the discovered sets' scores. Necessity therefore separates the discovery settings, while steering at this strength does not. We turn these findings into a protocol for reporting visual-head discoveries.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 39 of its 43 references that are in the map. First posted on arXiv 9 Oct 2026.

Builds on

39 of 43 references.

Closest ancestors · 11

Same direction, newest first

  1. 2026Gaze Heads: How VLMs Look at What They Describe12 Jun
  2. 2025Interpreting Attention Heads for Image-to-Text Information Flow in Large Vision-Language Models22 Sep
  3. 2025Your Large Vision-Language Model Only Needs A Few Attention Heads For Visual Grounding8 Mar
  4. 2024Cross-modal Information Flow in Multimodal Large Language Models27 Nov
  5. 2024Towards Interpreting Visual Information Processing in Vision-Language Models9 Oct
  6. 2024What Do VLMs NOTICE? A Mechanistic Interpretability Pipeline for Gaussian-Noise-free Text-Image Corruption and Evaluation24 Jun
  7. 2024Understanding Information Storage and Transfer in Multi-modal Large Language Models6 Jun
  8. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?29 Mar
  9. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
  10. 2023Teaching CLIP to Count to Ten23 Feb
  11. 2022Flamingo: a Visual Language Model for Few-Shot Learning29 Apr

Other references · 28

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  4. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  5. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  6. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 856
  7. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  8. 2019Are Sixteen Heads Really Better than One?cited by 816
  9. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
  10. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 528
  11. 2021Causal Abstractions of Neural Networkscited by 422
  12. 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Modelscited by 311
16 more
  1. 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 243
  2. 2024Retrieval Head Mechanistically Explains Long-Context Factualitycited by 232
  3. 2024How to use and interpret activation patchingcited by 214
  4. 2023Attribution Patching Outperforms Automated Circuit Discoverycited by 209
  5. 2023Localizing Model Behavior with Path Patchingcited by 195
  6. 2025See What You Are Told: Visual Attention Sink in Large Multimodal Modelscited by 132
  7. 2024AtP*: An efficient and scalable method for localizing LLM behaviour to componentscited by 111
  8. 2021Learning to Predict Visual Attributes in the Wildcited by 96
  9. 2023Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMscited by 91
  10. 2025MIB: A Mechanistic Interpretability Benchmarkcited by 56
  11. 2025Spatial457: A Diagnostic Benchmark for 6D Spatial Reasoning of Large Multimodal Modelscited by 38
  12. 2024Unveiling Visual Perception in Language Models: An Attention Head Analysis Approachcited by 35
  13. 2024Transformer Circuit Faithfulness Metrics are not Robustcited by 22
  14. 2026Mechanistic Insights into Functional Sparsity in Multimodal LLMs via CoRe Headscited by 4
  15. 2026Targeting the Attention Heads Behind Object Hallucination in LLaVAcited by 1
  16. 2026Retrieval Heads Meet Vision: Uncovering How VLMs Locate and Extract Visual Informationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Seeing without Looking: Do Vision-Language Benchmarks Really Test Vision?
  2. 2026Where Reliability Lives in Vision-Language Models: A Mechanistic Study of Attention, Hidden States, and Causal Circuits
  3. 2026Mirage Probes: How Vision Models Fake Visual Understanding
  4. 2025Examining Vision Language Models through Multi-dimensional Experiments with Vision and Text Features
  5. 2026Seeing is not Enough: Vision-Language Models Perceive Evidence but Fail to Act
  6. 2026Visuals Lie, Consistency Speaks: Disentangling Spatial Attention from Reliability in Vision-Language Models
  7. 2025Seeing but Not Believing: Probing the Disconnect Between Visual Attention and Answer Correctness in VLMs
  8. 2025Investigating The Functional Roles of Attention Heads in Vision Language Models: Evidence for Reasoning Modules