mlsanityData to 14 Oct 2026

Finding and Editing Multi-Modal Neurons in Pre-Trained Transformers

arXiv 2311.07470 ↗First posted 13 Nov 2023cs.CLcited by 36

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Understanding the internal mechanisms by which multi-modal large language models (LLMs) interpret different modalities and integrate cross-modal representations is becoming increasingly critical for continuous improvements in both academia and industry. In this paper, we propose a novel method to identify key neurons for interpretability -- how multi-modal LLMs bridge visual and textual concepts for captioning. Our method improves conventional works upon efficiency and applied range by removing needs of costly gradient computation. Based on those identified neurons, we further design a multi-modal knowledge editing method, beneficial to mitigate sensitive words or hallucination. For rationale of our design, we provide theoretical assumption. For empirical evaluation, we have conducted extensive quantitative and qualitative experiments. The results not only validate the effectiveness of our methods, but also offer insightful findings that highlight three key properties of multi-modal neurons: sensitivity, specificity and causal-effect, to shed light for future research.

Key facts. Cited by 36 papers in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 36 of its 62 references that are in the map. First posted on arXiv 13 Nov 2023.

Builds on

36 of 62 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration7 Nov

Other references · 35

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  7. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  8. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  9. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  10. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  11. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  12. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
23 more
  1. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
  2. 2020Concept Bottleneck Modelscited by 1,032
  3. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,031
  4. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 1,003
  5. 2022Mass-Editing Memory in a Transformercited by 1,001
  6. 2022In-context Learning and Induction Headscited by 964
  7. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 856
  8. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 708
  9. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 691
  10. 2019MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distancecited by 418
  11. 2019Are Transformers universal approximators of sequence-to-sequence functions?cited by 406
  12. 2019Linguistic Knowledge and Transferability of Contextual Representationscited by 383
  13. 2020Understanding the Role of Individual Units in a Deep Neural Networkcited by 255
  14. 2019Probing Neural Network Comprehension of Natural Language Argumentscited by 214
  15. 2018Dissecting Contextual Word Embeddings: Architecture and Representationcited by 146
  16. 2022Natural Language Descriptions of Deep Visual Featurescited by 137
  17. 2020Self-Attention Attribution: Interpreting Information Interactions Inside Transformercited by 125
  18. 2023Language Models Implement Simple Word2Vec-style Vector Arithmeticcited by 98
  19. 2021Deconfounded Video Moment Retrieval with Causal Interventioncited by 84
  20. 2019Improving Transformer Models by Reordering their Sublayerscited by 52
  21. 2023Multimodal Neurons in Pretrained Text-Only Transformerscited by 49
  22. 2020Tree-Augmented Cross-Modal Encoding for Complex-Query Video Retrievalcited by 47
  23. 2022What Are You Token About? Dense Retrieval as Distributions Over the Vocabularycited by 28

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 18; 2025: 14; 2026: 4.

36 papers.

Sort:

By area:

  1. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 163
  2. 2024Visual In-Context Learning for Large Vision-Language Modelscited by 128
  3. 2024Detoxifying Large Language Models via Knowledge Editingcited by 87
  4. 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Surveycited by 52
  5. 2024Knowledge Mechanisms in Large Language Models: A Survey and Perspectivecited by 51
  6. 2024A Concept-Based Explainability Framework for Large Multimodal Modelscited by 39
  7. 2025A Survey on Mechanistic Interpretability for Multi-Modal Foundation Modelscited by 34
  8. 2024MMNeuron: Discovering Neuron-Level Domain-Specific Interpretation in Multimodal Large Language Modelcited by 26
  9. 2024Concept-skill Transferability-based Data Selection for Large Vision-Language Modelscited by 24
  10. 2024Implicit Multimodal Alignment: On the Generalization of Frozen LLMs to Multimodal Inputscited by 23
  11. 2025Learning to See Before Seeing: Demystifying LLM Visual Priors from Language Pre-trainingcited by 23
  12. 2024Cross-Modal Projection in Multimodal LLMs Doesn't Really Project Visual Attributes to Textual Spacecited by 22
24 more
  1. 2024Neuron-Level Sequential Editing for Large Language Modelscited by 20
  2. 2024Towards Unified Multimodal Editing with Enhanced Knowledge Collaborationcited by 18
  3. 2024InstructEdit: Instruction-based Knowledge Editing for Large Language Modelscited by 17
  4. 2025Analyzing Finetuning Representation Shift for Multimodal LLMs Steeringcited by 16
  5. 2024Attribution Analysis Meets Model Editing: Advancing Knowledge Correction in Vision Language Models with VisEditcited by 14
  6. 2025VL-SAE: Interpreting and Enhancing Vision-Language Alignment with a Unified Concept Setcited by 13
  7. 2025Model Utility Law: Evaluating LLMs beyond Performance through Mechanism Interpretable Metriccited by 11
  8. 2025Where Culture Fades: Revealing the Cultural Gap in Text-to-Image Generationcited by 11
  9. 2025Do LLMs Signal When They're Right? Evidence from Neuron Agreementcited by 10
  10. 2024Visual-Oriented Fine-Grained Knowledge Editing for MultiModal Large Language Modelscited by 9
  11. 2025QR-LoRA: Efficient and Disentangled Fine-tuning via QR Decomposition for Customized Generationcited by 9
  12. 2025CMMLoc: Advancing Text-to-PointCloud Localization with Cauchy-Mixture-Model Based Frameworkcited by 8
  13. 2026BalDRO: A Distributionally Robust Optimization based Framework for Large Language Model Unlearningcited by 8
  14. 2024MINER: Mining the Underlying Pattern of Modality-Specific Neurons in Multimodal Large Language Modelscited by 7
  15. 2026Instruction Anchor: Dissecting the Mechanistic Dynamics of Modality Arbitrationcited by 6
  16. 2025From Concepts to Components: Concept-Agnostic Attention Module Discovery in Transformerscited by 4
  17. 2025Deciphering Functions of Neurons in Vision-Language Modelscited by 3
  18. 2025Identifying Multi-modal Knowledge Neurons in Pretrained Transformers via Two-stage Filteringcited by 3
  19. 2025Seeing It or Not? Interpretable Vision-aware Latent Steering to Mitigate Object Hallucinationscited by 3
  20. 2024Advancing Prompt Learning through an External Layercited by 2
  21. 2025Self-Aware Safety Augmentation: Leveraging Internal Semantic Understanding to Enhance Safety in Vision-Language Modelscited by 2
  22. 2026Neuron-Anchored Rule Extraction for Large Language Models via Contrastive Hierarchical Ablationcited by 2
  23. 2026Generalizable Multimodal Large Language Model Editing via Invariant Trajectory Learningcited by 1
  24. 2024If you can describe it, they can see it: Cross-Modal Learning of Visual Concepts from Textual Descriptionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Correct When Paired, Wrong When Split: Decoupling and Editing Modality-Specific Neurons in MLLMs
  2. 2025Towards LLM-Centric Multimodal Fusion: A Survey on Integration Strategies and Techniques
  3. 2024Large Multi-modal Models Can Interpret Features in Large Multi-modal Models
  4. 2026Do LLMs and VLMs Share Neurons for Inference? Evidence and Mechanisms of Cross-Modal Transfer
  5. 2024ModaVerse: Efficiently Transforming Modalities with LLMs
  6. 2026MNAFT: modality neuron-aware fine-tuning of multimodal large language models for image translation
  7. 2025HaploVL: A Single-Transformer Baseline for Multi-Modal Understanding
  8. 2024A Review of Multi-Modal Large Language and Vision Models