mlsanityData to 7 Oct 2026

What Do VLMs NOTICE? A Mechanistic Interpretability Pipeline for Gaussian-Noise-free Text-Image Corruption and Evaluation

arXiv 2406.16320 ↗First posted 24 Jun 2024cs.CLcited by 29

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Vision-Language Models (VLMs) have gained community-spanning prominence due to their ability to integrate visual and textual inputs to perform complex tasks. Despite their success, the internal decision-making processes of these models remain opaque, posing challenges in high-stakes applications. To address this, we introduce NOTICE, the first Noise-free Text-Image Corruption and Evaluation pipeline for mechanistic interpretability in VLMs. NOTICE incorporates a Semantic Minimal Pairs (SMP) framework for image corruption and Symmetric Token Replacement (STR) for text. This approach enables semantically meaningful causal mediation analysis for both modalities, providing a robust method for analyzing multimodal integration within models like BLIP. Our experiments on the SVO-Probes, MIT-States, and Facial Expression Recognition datasets reveal crucial insights into VLM decision-making, identifying the significant role of middle-layer cross-attention heads. Further, we uncover a set of ``universal cross-attention heads'' that consistently contribute across tasks and modalities, each performing distinct functions such as implicit image segmentation, object inhibition, and outlier inhibition. This work paves the way for more transparent and interpretable multimodal systems.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 38 of its 61 references that are in the map. First posted on arXiv 24 Jun 2024.

Builds on

38 of 61 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond24 Aug
  2. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation28 Jan

Other references · 36

  1. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  2. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  3. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  4. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  5. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  6. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  7. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,291
  8. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
  9. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  10. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  11. 2019Attention is not not Explanationcited by 629
  12. 2022When and why vision-language models behave like bags-of-words, and what to do about it?cited by 603
24 more
  1. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 452
  2. 2022Multimodal Learning with Transformers: A Surveycited by 399
  3. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 328
  4. 2013Challenges in Representation Learning: A report on three machine learning contestscited by 301
  5. 2023Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioningcited by 263
  6. 2021IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoningcited by 262
  7. 2019Assessing BERT's Syntactic Abilitiescited by 191
  8. 2023A Toy Model of Universality: Reverse Engineering How Networks Learn Group Operationscited by 141
  9. 2021Probing Image-Language Transformers for Verb Understandingcited by 109
  10. 2023Circuit Component Reuse Across Tasks in Transformer Language Modelscited by 105
  11. 2018Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Informationcited by 97
  12. 2020Analyzing Individual Neurons in Pre-trained Language Modelscited by 97
  13. 2017FOIL it! Find One mismatch between Image and Language captioncited by 95
  14. 2023Successor Heads: Recurring, Interpretable Attention Heads In The Wildcited by 77
  15. 2021Vision-and-Language or Vision-for-Language? On Cross-Modal Influence in Multimodal Transformerscited by 71
  16. 2023Towards Vision-Language Mechanistic Interpretability: A Causal Tracing Tool for BLIPcited by 62
  17. 2020Does my multimodal model learn cross-modal interactions? It's harder to tell than you might think!cited by 61
  18. 2017ShapeWorld - A new test methodology for multimodal language understandingcited by 37
  19. 2022MultiViz: Towards Visualizing and Understanding Multimodal Modelscited by 36
  20. 2020Seeing past words: Testing the cross-modal capabilities of pretrained V&L models on counting taskscited by 34
  21. 2015Sentiment of Emojiscited by 31
  22. 2023Analyzing Vision Transformers for Image Classification in Class Embedding Spacecited by 26
  23. 2023Outlier Dimensions Encode Task-Specific Knowledgecited by 15
  24. 2023LLMs as Visual Explainers: Advancing Image Classification with Evolving Visual Descriptionscited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 11; 2026: 18.

29 papers.

Sort:

By area:

  1. 2025Same Task, Different Circuits: Disentangling Modality-Specific Mechanisms in VLMscited by 35
  2. 2025Overcoming Sparsity Artifacts in Crosscoders to Interpret Chat-Tuningcited by 31
  3. 2025How Do Vision-Language Models Process Conflicting Information Across Modalities?cited by 22
  4. 2025Forgotten Polygons: Multimodal Large Language Models are Shape-Blindcited by 16
  5. 2025How Multimodal LLMs Solve Image Tasks: A Lens on Visual Grounding, Task Reasoning, and Answer Decodingcited by 13
  6. 2025Visual symbolic mechanisms: Emergent symbol processing in vision language modelscited by 12
  7. 2025V-SEAM: Visual Semantic Editing and Attention Modulating for Causal Interpretability of Vision-Language Modelscited by 7
  8. 2026Interpretable Debiasing of Vision-Language Models for Social Fairnesscited by 6
  9. 2026Are Audio-Language Models Listening? Audio-Specialist Heads for Adaptive Audio Steeringcited by 4
  10. 2026Causal Tracing of Audio-Text Fusion in Large Audio Language Modelscited by 4
  11. 2025Diagnosing Bottlenecks in Data Visualization Understanding by Vision-Language Modelscited by 2
  12. 2026Probing Visual Concepts in Lightweight Vision-Language Models for Automated Drivingcited by 2
17 more
  1. 2026Gaze Heads: How VLMs Look at What They Describecited by 2
  2. 2025Common Objects Out of Context (COOCo): Investigating Multimodal Context and Semantic Scene Violations in Referential Communicationcited by 1
  3. 2025MIMIC: Multimodal Inversion for Model Interpretation and Conceptualizationcited by 1
  4. 2026What Do They See? Interpreting Complex Road Scenarios Through the Eyes of Vision-Language-Action Models for Safe and Trustworthy Autonomous Vehicle Learningcited by 1
  5. 2025Beyond the Linear Separability Ceiling: Aligning Representations in VLMscited by 0
  6. 2026Beyond Dominant Patches: Spatial Credit Redistribution For Grounded Vision-Language Modelscited by 0
  7. 2026Benchmarking and Mechanistic Analysis of Vision-Language Models for Cross-Depiction Assembly Instruction Alignmentcited by 0
  8. 2026Latent Anomaly Knowledge Excavation: Unveiling Sparse Sensitive Neurons in Vision-Language Modelscited by 0
  9. 2026Multimodal QUD: Inquisitive Questions from Scientific Figurescited by 0
  10. 2026Interpreting and Enhancing Emotional Circuits in Large Vision-Language Models via Cross-Modal Information Flowcited by 0
  11. 2026When Structured Sparse Autoencoders Learn Consistent Concepts Across Modalitiescited by 0
  12. 2026How Do VLMs Fail? Vision-Operation Misalignment in Compositional VQAcited by 0
  13. 2026ENTRAP-VL: A Taxonomic Probe for Dual Contextual Entrainment in Vision-Language Modelscited by 0
  14. 2026Prior Directions: Why GUI Grounding Gets Locked in the Pastcited by 0
  15. 2026From Retrieval to Recognition:How Vision--Language Models Become OCR Specialistscited by 0
  16. 2026Who Is Left of Whom? Tracing Spatial Evidence and Role Binding in Relative-Position Reasoningcited by 0
  17. 2026CRAFT: Causal Responsibility and Failure Tracing in Medical Vision Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Mixed Signals: Decoding VLMs' Reasoning and Underlying Bias in Vision-Language Conflict
  2. 2025From Behavioral Performance to Internal Competence: Interpreting Vision-Language Models with VLM-Lens
  3. 2026ReViCo: Unveiling the Limitations of VLMs in Visual Text Understanding via Error Correction
  4. 2026Measuring Cross-Modal Synergy: A Benchmark for VLM Explainability
  5. 2024Towards Interpreting Visual Information Processing in Vision-Language Models
  6. 2025Better Reasoning with Less Data: Enhancing VLMs Through Unified Modality Scoring
  7. 2025Analysing the Robustness of Vision-Language-Models to Common Corruptions
  8. 2023Synthesize, Diagnose, and Optimize: Towards Fine-Grained Vision-Language Understanding