mlsanityData to 7 Oct 2026

Multimodal Neurons in Pretrained Text-Only Transformers

arXiv 2308.01544 ↗First posted 3 Aug 2023cs.CV cs.CLcited by 49

Not on the map. Why?

Cited by 30 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Language models demonstrate remarkable capacity to generalize representations learned in one modality to downstream tasks in other modalities. Can we trace this ability to individual neurons? We study the case where a frozen text transformer is augmented with vision using a self-supervised visual encoder and a single linear projection learned on an image-to-text task. Outputs of the projection layer are not immediately decodable into language describing image content; instead, we find that translation between modalities occurs deeper within the transformer. We introduce a procedure for identifying "multimodal neurons" that convert visual representations into corresponding text, and decoding the concepts they inject into the model's residual stream. In a series of experiments, we show that multimodal neurons operate on specific visual concepts across inputs, and have a systematic causal effect on image captioning.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Builds on 31 of its 49 references that are in the map. First posted on arXiv 3 Aug 2023.

Builds on

31 of 49 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  7. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  8. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  9. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  10. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  11. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  12. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
19 more
  1. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  2. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  3. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  4. 2022Mass-Editing Memory in a Transformercited by 976
  5. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 703
  6. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 657
  7. 2021Knowledge Neurons in Pretrained Transformerscited by 590
  8. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 417
  9. 2020Understanding the Role of Individual Units in a Deep Neural Networkcited by 254
  10. 2023Large Language Models as General Pattern Machinescited by 196
  11. 2021Pretrained Transformers as Universal Computation Enginescited by 162
  12. 2022Linearly Mapping from Image to Text Spacecited by 139
  13. 2022Natural Language Descriptions of Deep Visual Featurescited by 136
  14. 2017Interpreting Deep Visual Representations via Network Dissectioncited by 103
  15. 2022Language Models are General-Purpose Interfacescited by 96
  16. 2021MAGMA -- Multimodal Augmentation of Generative Models through Adapter-based Finetuningcited by 85
  17. 2022Plansformer: Generating Symbolic Plans using Transformerscited by 41
  18. 2022LM-Debugger: An Interactive Tool for Inspection and Intervention in Transformer-Based Language Modelscited by 28
  19. 2022What Are You Token About? Dense Retrieval as Distributions Over the Vocabularycited by 28

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 17; 2025: 16; 2026: 11.

49 papers.

Sort:

By area:

  1. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 225
  2. 2024Visual In-Context Learning for Large Vision-Language Modelscited by 124
  3. 2024Towards Interpreting Visual Information Processing in Vision-Language Modelscited by 115
  4. 2024Interpreting and Editing Vision-Language Representations to Mitigate Hallucinationscited by 103
  5. 2024Cross-modal Information Flow in Multimodal Large Language Modelscited by 65
  6. 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Surveycited by 52
  7. 2024Knowledge Mechanisms in Large Language Models: A Survey and Perspectivecited by 50
  8. 2023Frozen Transformers in Language Models Are Effective Visual Encoder Layerscited by 39
  9. 2024A Concept-Based Explainability Framework for Large Multimodal Modelscited by 39
  10. 2025Vision Transformers Don't Need Trained Registerscited by 39
  11. 2023Finding and Editing Multi-Modal Neurons in Pre-Trained Transformerscited by 36
  12. 2025Visual Large Language Models for Generalized and Specialized Applicationscited by 35
37 more
  1. 2025A Survey on Mechanistic Interpretability for Multi-Modal Foundation Modelscited by 34
  2. 2024MMNeuron: Discovering Neuron-Level Domain-Specific Interpretation in Multimodal Large Language Modelcited by 26
  3. 2024Concept-skill Transferability-based Data Selection for Large Vision-Language Modelscited by 24
  4. 2025How Visual Representations Map to Language Feature Space in Multimodal LLMscited by 24
  5. 2025TACO: Enhancing Multimodal In-context Learning via Task Mapping-Guided Sequence Configurationcited by 23
  6. 2025Learning to See Before Seeing: Demystifying LLM Visual Priors from Language Pre-trainingcited by 23
  7. 2024Implicit Multimodal Alignment: On the Generalization of Frozen LLMs to Multimodal Inputscited by 22
  8. 2024Cross-Modal Projection in Multimodal LLMs Doesn't Really Project Visual Attributes to Textual Spacecited by 21
  9. 2024Neuron-Level Sequential Editing for Large Language Modelscited by 18
  10. 2025Analyzing Finetuning Representation Shift for Multimodal LLMs Steeringcited by 15
  11. 2024Attribution Analysis Meets Model Editing: Advancing Knowledge Correction in Vision Language Models with VisEditcited by 14
  12. 2023Object Recognition as Next Token Predictioncited by 12
  13. 2024Automatic Discovery of Visual Circuitscited by 12
  14. 2025VL-SAE: Interpreting and Enhancing Vision-Language Alignment with a Unified Concept Setcited by 12
  15. 2025When Seeing Overrides Knowing: Disentangling Knowledge Conflicts in Vision-Language Modelscited by 11
  16. 2023Multimodal Attention Merging for Improved Speech Recognition and Audio Event Classificationcited by 10
  17. 2026DNA: Uncovering Universal Latent Forgery Knowledgecited by 10
  18. 2026Arbitration Failure, Not Perceptual Blindness: How Vision-Language Models Resolve Visual-Linguistic Conflictscited by 9
  19. 2025Where Culture Fades: Revealing the Cultural Gap in Text-to-Image Generationcited by 8
  20. 2024MINER: Mining the Underlying Pattern of Modality-Specific Neurons in Multimodal Large Language Modelscited by 7
  21. 2025Locate-then-Merge: Neuron-Level Parameter Fusion for Mitigating Catastrophic Forgetting in Multimodal LLMscited by 7
  22. 2025Merge then Realign: Simple and Effective Modality-Incremental Continual Learning for Multimodal LLMscited by 6
  23. 2025Too Late to Recall: Explaining the Two-Hop Problem in Multimodal Knowledge Retrievalcited by 6
  24. 2025Advancing Multimodal In-Context Learning in Large Vision-Language Models with Task-aware Demonstrationscited by 5
  25. 2025An Empirical Study on How Video-LLMs Answer Video Questionscited by 4
  26. 2026A Comprehensive Information-Decomposition Analysis of Large Vision-Language Modelscited by 3
  27. 2025Self-Aware Safety Augmentation: Leveraging Internal Semantic Understanding to Enhance Safety in Vision-Language Modelscited by 2
  28. 2023Representing visual classification as a linear combination of wordscited by 1
  29. 2026Towards Understanding Multimodal Fine-Tuning: Spatial Featurescited by 1
  30. 2026Nano-EmoX: Unifying Multimodal Emotional Intelligence from Perception to Empathycited by 1
  31. 2024If you can describe it, they can see it: Cross-Modal Learning of Visual Concepts from Textual Descriptionscited by 0
  32. 2026Mind the Gap No More: Achieving Zero-Gap Multimodal Integration via One Tokenizercited by 0
  33. 2026V-FIND: Revealing the Intrinsic Forgery Knowledge Encoded in Video Forgery Detectorscited by 0
  34. 2026Multimodal Model Diffing for Feature Discovery and Controlcited by 0
  35. 2026Using OCR Heads to Verbalize Image Semanticscited by 0
  36. 2026Can Circuit Alignment Predict OOD Generalization?cited by 0
  37. 2026Encoded but Disconnected: Decomposing Vision-Language Model Failures under a Patching Nullcited by 0