mlsanityData to 14 Oct 2026

Deciphering Functions of Neurons in Vision-Language Models

arXiv 2502.18485 ↗First posted 10 Feb 2025q-bio.NC cs.CVcited by 3

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

The burgeoning growth of open-sourced vision-language models (VLMs) has catalyzed a plethora of applications across diverse domains. Ensuring the transparency and interpretability of these models is critical for fostering trustworthy and responsible AI systems. In this study, our objective is to delve into the internals of VLMs to interpret the functions of individual neurons. We observe the activations of neurons with respects to the input visual tokens and text tokens, and reveal some interesting findings. Particularly, we found that there are neurons responsible for only visual or text information, or both, respectively, which we refer to them as visual neurons, text neurons, and multi-modal neurons, respectively. We build a framework that automates the explanation of neurons with the assistant of GPT-4o. Meanwhile, for visual neurons, we propose an activation simulator to assess the reliability of the explanations for visual neurons. System statistical analyses on top of one representative VLM of LLaVA, uncover the behaviors/characteristics of different categories of neurons.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 19 of its 29 references that are in the map. First posted on arXiv 10 Feb 2025.

Builds on

19 of 29 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024PaliGemma: A versatile 3B VLM for transfer10 Jul

Other references · 18

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2023Visual Instruction Tuningcited by 7,869
  3. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  4. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  5. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  6. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
  7. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
  8. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 708
  9. 2023A Comprehensive Overview of Large Language Modelscited by 648
  10. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
  11. 2022CLIP-Dissect: Automatic Description of Neuron Representations in Deep Vision Networkscited by 147
  12. 2022Natural Language Descriptions of Deep Visual Featurescited by 137
6 more
  1. 2024Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Modelscited by 126
  2. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 118
  3. 2024Rethinking Interpretability in the Era of Large Language Modelscited by 111
  4. 2023Finding and Editing Multi-Modal Neurons in Pre-Trained Transformerscited by 36
  5. 2023Labeling Neural Representations with Inverse Recognitioncited by 32
  6. 2024Interpreting Neurons in Deep Vision Networks with Language Modelscited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Language Models Can Explain Visual Features via Steeringcited by 2
  2. 2026When Structured Sparse Autoencoders Learn Consistent Concepts Across Modalitiescited by 1
  3. 2026Emotion Across Speech and Faces: Shared Affective Mechanisms in Multimodal Foundation Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026NeuronEye: Query-Guided Visual Concept Activation for Vision-Language Reasoning
  2. 2026Do LLMs and VLMs Share Neurons for Inference? Evidence and Mechanisms of Cross-Modal Transfer
  3. 2026Mind the Spike: Mechanisms and Brittleness of Visual Massive Activations in Large Vision-Language Models
  4. 2026Structural Graph Probing of Vision-Language Models
  5. 2024Towards Interpreting Visual Information Processing in Vision-Language Models
  6. 2025Synthesizing Visual Concepts as Vision-Language Programs
  7. 2024MMNeuron: Discovering Neuron-Level Domain-Specific Interpretation in Multimodal Large Language Model
  8. 2025Sparse Neuron Ablation Triggers Catastrophic Collapse of the Language Core in Large Vision-Language Models