mlsanityData to 14 Oct 2026

MMNeuron: Discovering Neuron-Level Domain-Specific Interpretation in Multimodal Large Language Model

arXiv 2406.11193 ↗First posted 17 Jun 2024cs.CLcited by 26

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Projecting visual features into word embedding space has become a significant fusion strategy adopted by Multimodal Large Language Models (MLLMs). However, its internal mechanisms have yet to be explored. Inspired by multilingual research, we identify domain-specific neurons in multimodal large language models. Specifically, we investigate the distribution of domain-specific neurons and the mechanism of how MLLMs process features from diverse domains. Furthermore, we propose a three-stage mechanism for language model modules in MLLMs when handling projected image features, and verify this hypothesis using logit lens. Extensive experiments indicate that while current MLLMs exhibit Visual Question Answering (VQA) capability, they may not fully utilize domain-specific information. Manipulating domain-specific neurons properly will result in a 10% change of accuracy at most, shedding light on the development of cross-domain, all-encompassing MLLMs in the future. The source code is available at https://github.com/Z1zs/MMNeuron.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 45 of its 66 references that are in the map. First posted on arXiv 17 Jun 2024.

Builds on

45 of 66 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Finding and Editing Multi-Modal Neurons in Pre-Trained Transformers13 Nov
  2. 2023Improved Baselines with Visual Instruction Tuning5 Oct
  3. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning11 May

Other references · 42

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  6. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  7. 2015VQA: Visual Question Answeringcited by 2,506
  8. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  9. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  10. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
  11. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  12. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
30 more
  1. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 708
  2. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 675
  3. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
  4. 2019Scene Text Visual Question Answeringcited by 331
  5. 2023GeoChat: Grounded Large Vision-Language Model for Remote Sensingcited by 328
  6. 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 281
  7. 2020RSVQA: Visual Question Answering for Remote Sensing Datacited by 203
  8. 2023Interpreting CLIP's Image Representation via Text-Based Decompositioncited by 196
  9. 2020Compositional Explanations of Neuronscited by 172
  10. 2024How do Large Language Models Handle Multilingualism?cited by 150
  11. 2017Learning to Generate Reviews and Discovering Sentimentcited by 148
  12. 2022CLIP-Dissect: Automatic Description of Neuron Representations in Deep Vision Networkscited by 147
  13. 2023LingoQA: Visual Question Answering for Autonomous Drivingcited by 145
  14. 2022Linearly Mapping from Image to Text Spacecited by 142
  15. 2022Natural Language Descriptions of Deep Visual Featurescited by 137
  16. 2024Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Modelscited by 126
  17. 2018Identifying and Controlling Important Neurons in Neural Machine Translationcited by 111
  18. 2023Quilt-LLaVA: Visual Instruction Tuning by Extracting Localized Narratives from Open-Source Histopathology Videoscited by 86
  19. 2024On the Multilingual Ability of Decoder-based Pre-trained Language Models: Finding and Controlling Language-Specific Neuronscited by 83
  20. 2021Neuron-level Interpretation of Deep NLP Models: A Surveycited by 81
  21. 2022BridgeTower: Building Bridges Between Encoders in Vision-Language Representation Learningcited by 56
  22. 2023Journey to the Center of the Knowledge Neurons: Discoveries of Language-Independent Knowledge Neurons and Degenerate Knowledge Neuronscited by 56
  23. 2024ConceptPrune: Concept Editing in Diffusion Models via Skilled Neuron Pruningcited by 55
  24. 2022VL-InterpreT: An Interactive Visualization Tool for Interpreting Vision-Language Transformerscited by 53
  25. 2023Multimodal Neurons in Pretrained Text-Only Transformerscited by 49
  26. 2024What does the Knowledge Neuron Thesis Have to do with Knowledge?cited by 49
  27. 2024Tracing the Roots of Facts in Multilingual Language Models: Independent, Shared, and Transferred Knowledgecited by 14
  28. 2024Interpreting Neurons in Deep Vision Networks with Language Modelscited by 10
  29. 2024Revealing the Parallel Multilingual Learning within Large Language Modelscited by 8
  30. 2024Neuron-based Multifractal Analysis of Neuron Interaction Dynamics in Large Modelscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 13; 2026: 8.

26 papers.

Sort:

By area:

  1. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 116
  2. 2024Interpreting and Editing Vision-Language Representations to Mitigate Hallucinationscited by 104
  3. 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Surveycited by 52
  4. 2024Mitigating Modality Prior-Induced Hallucinations in Multimodal Large Language Models via Deciphering Attention Causalitycited by 42
  5. 2024ErrorRadar: Benchmarking Complex Mathematical Reasoning of Multimodal Large Language Models Via Error Detectioncited by 37
  6. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 27
  7. 2025DocPruner: A Storage-Efficient Framework for Multi-Vector Visual Document Retrieval via Adaptive Patch-Level Embedding Pruningcited by 20
  8. 2025AudioLens: A Closer Look at Auditory Attribute Perception of Large Audio-Language Modelscited by 18
  9. 2025Visual Representations inside the Language Modelcited by 15
  10. 2025GM-PRM: A Generative Multimodal Process Reward Model for Multimodal Mathematical Reasoningcited by 14
  11. 2025How Multimodal LLMs Solve Image Tasks: A Lens on Visual Grounding, Task Reasoning, and Answer Decodingcited by 13
  12. 2025MAC: A Live Benchmark for Multimodal Large Language Models in Scientific Understandingcited by 12
14 more
  1. 2025Where Culture Fades: Revealing the Cultural Gap in Text-to-Image Generationcited by 11
  2. 2025PhysicsArena: The First Multimodal Physics Reasoning Benchmark Exploring Variable, Process, and Solution Dimensionscited by 9
  3. 2024MINER: Mining the Underlying Pattern of Modality-Specific Neurons in Multimodal Large Language Modelscited by 7
  4. 2025Connector-S: A Survey of Connectors in Multi-modal Large Language Modelscited by 7
  5. 2026Interpretable Debiasing of Vision-Language Models for Social Fairnesscited by 6
  6. 2025Brain-Inspired Exploration of Functional Networks and Key Neurons in Large Language Modelscited by 5
  7. 2026Latent Anomaly Knowledge Excavation: Unveiling Sparse Sensitive Neurons in Vision-Language Modelscited by 3
  8. 2026Null Space Constrained Contrastive Visual Forgetting for MLLM Unlearningcited by 3
  9. 2026Neuron-Level Emotion Control in Speech-Generative Large Audio-Language Modelscited by 2
  10. 2025Decoding the Multimodal Maze: A Systematic Review on the Adoption of Explainability in Multimodal Attention-based Modelscited by 0
  11. 2025TRANSPORTER: Transferring Visual Semantics from VLM Manifoldscited by 0
  12. 2026Concept-Enhanced Multimodal RAG: Towards Interpretable and Accurate Radiology Report Generationcited by 0
  13. 2026Emotion Across Speech and Faces: Shared Affective Mechanisms in Multimodal Foundation Modelscited by 0
  14. 2026Who Is Left of Whom? Tracing Spatial Evidence and Role Binding in Relative-Position Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Cross-modal Information Flow in Multimodal Large Language Models
  2. 2025Language-Specific Layer Matters: Efficient Multilingual Enhancement for Large Vision-Language Models
  3. 2024Understanding Multimodal LLMs: the Mechanistic Interpretability of Llava in Visual Question Answering
  4. 2024Probing Multimodal Large Language Models for Global and Local Semantic Representations
  5. 2024The Revolution of Multimodal Large Language Models: A Survey
  6. 2024Efficient Multimodal Large Language Models: A Survey
  7. 2026MLLM-Microscope: Unlocking Hidden Structure Within Multimodal Large Language Models
  8. 2025Identifying Multi-modal Knowledge Neurons in Pretrained Transformers via Two-stage Filtering