mlsanityData to 14 Oct 2026

Understanding Multimodal LLMs: the Mechanistic Interpretability of Llava in Visual Question Answering

arXiv 2411.10950 ↗First posted 17 Nov 2024cs.CLcited by 11

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

also in 1 more
Abstract (arXiv)

Understanding the mechanisms behind Large Language Models (LLMs) is crucial for designing improved models and strategies. While recent studies have yielded valuable insights into the mechanisms of textual LLMs, the mechanisms of Multi-modal Large Language Models (MLLMs) remain underexplored. In this paper, we apply mechanistic interpretability methods to analyze the visual question answering (VQA) mechanisms in the first MLLM, Llava. We compare the mechanisms between VQA and textual QA (TQA) in color answering tasks and find that: a) VQA exhibits a mechanism similar to the in-context learning mechanism observed in TQA; b) the visual features exhibit significant interpretability when projecting the visual embeddings into the embedding space; and c) Llava enhances the existing capabilities of the corresponding textual LLM Vicuna during visual instruction tuning. Based on these findings, we develop an interpretability tool to help users and researchers identify important visual locations for final predictions, aiding in the understanding of visual hallucination. Our method demonstrates faster and more effective results compared to existing interpretability approaches. Code: urlhttps://github.com/zepingyu0512/llava-mechanism

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 42 of its 50 references that are in the map. First posted on arXiv 17 Nov 2024.

Builds on

42 of 50 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Understanding Information Storage and Transfer in Multi-modal Large Language Models6 Jun

Other references · 41

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2022Training language models to follow instructions with human feedbackcited by 15,971
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2023Visual Instruction Tuningcited by 7,869
  7. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  8. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
  9. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  10. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  11. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 735
  12. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
29 more
  1. 2021Knowledge Neurons in Pretrained Transformerscited by 601
  2. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 532
  3. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 528
  4. 2023Larger language models do in-context learning differentlycited by 394
  5. 2024Hallucination of Multimodal Large Language Models: A Surveycited by 388
  6. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 359
  7. 2024A Survey on Hallucination in Large Vision-Language Modelscited by 357
  8. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 334
  9. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Modelscited by 316
  10. 2024MM-LLMs: Recent Advances in MultiModal Large Language Modelscited by 278
  11. 2023Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learningcited by 236
  12. 2023How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language modelcited by 202
  13. 2023Interpreting CLIP's Image Representation via Text-Based Decompositioncited by 196
  14. 2023Gender bias and stereotypes in Large Language Modelscited by 187
  15. 2023LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examplescited by 177
  16. 2023Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchillacited by 149
  17. 2024Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Trackingcited by 149
  18. 2022Analyzing Transformers in Embedding Spacecited by 132
  19. 2023Circuit Component Reuse Across Tasks in Transformer Language Modelscited by 108
  20. 2023A Mechanistic Interpretation of Arithmetic Reasoning in Language Models using Causal Mediation Analysiscited by 89
  21. 2021Neuron-level Interpretation of Deep NLP Models: A Surveycited by 81
  22. 2023Is This the Subspace You Are Looking for? An Interpretability Illusion for Subspace Activation Patchingcited by 80
  23. 2023Successor Heads: Recurring, Interpretable Attention Heads In The Wildcited by 77
  24. 2023Can ChatGPT Replace Traditional KBQA Models? An In-depth Analysis of the Question Answering Performance of the GPT LLM Familycited by 71
  25. 2024Visual Hallucinations of Multi-modal Large Language Modelscited by 68
  26. 2023Neuron-Level Knowledge Attribution in Large Language Modelscited by 37
  27. 2024Interpreting Arithmetic Mechanism in Large Language Models through Comparative Neuron Analysiscited by 31
  28. 2023Causal Interpretation of Self-Attention in Pre-Trained Transformerscited by 30
  29. 2024How do Large Language Models Learn In-Context? Query and Key Matrices of In-Context Heads are Two Towers for Metric Learningcited by 18

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 4.

11 papers.

  1. 2025A Survey of Context Engineering for Large Language Modelscited by 126
  2. 2025Same Task, Different Circuits: Disentangling Modality-Specific Mechanisms in VLMscited by 35
  3. 2026Linear Mechanisms for Spatiotemporal Reasoning in Vision Language Modelscited by 14
  4. 2025Map the Flow: Revealing Hidden Pathways of Information in VideoLLMscited by 10
  5. 2025Intervene-All-Paths: Unified Mitigation of LVLM Hallucinations across Alignment Formatscited by 6
  6. 2026Instruction Anchor: Dissecting the Mechanistic Dynamics of Modality Arbitrationcited by 6
  7. 2025REMA: A Unified Reasoning Manifold Framework for Interpreting Large Language Modelcited by 5
  8. 2026Null Space Constrained Contrastive Visual Forgetting for MLLM Unlearningcited by 3
  9. 2025Robust MLLM Unlearning via Visual Knowledge Distillationcited by 1
  10. 2025Interpret, prune and distill Donut : towards lightweight VLMs for VQA on documentcited by 0
  11. 2026Multi-Modal LLM based Image Captioning in ICT: Bridging the Gap Between General and Industry Domaincited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025How Multimodal LLMs Solve Image Tasks: A Lens on Visual Grounding, Task Reasoning, and Answer Decoding
  2. 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey
  3. 2023Dynamic Clue Bottlenecks: Towards Interpretable-by-Design Visual Question Answering
  4. 2025Language Models Can See Better: Visual Contrastive Decoding For LLM Multimodal Reasoning
  5. 2025Evaluating Variance in Visual Question Answering Benchmarks
  6. 2024Visual Question Answering Instruction: Unlocking Multimodal Large Language Model To Domain-Specific Visual Multitasks
  7. 2025Where do Large Vision-Language Models Look at when Answering Questions?
  8. 2024Cross-modal Information Flow in Multimodal Large Language Models