mlsanityData to 7 Oct 2026

Large Multi-modal Models Can Interpret Features in Large Multi-modal Models

arXiv 2411.14982 ↗First posted 22 Nov 2024cs.CV cs.CLcited by 23

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Recent advances in Large Multimodal Models (LMMs) lead to significant breakthroughs in both academia and industry. One question that arises is how we, as humans, can understand their internal neural representations. This paper takes an initial step towards addressing this question by presenting a versatile framework to identify and interpret the semantics within LMMs. Specifically, 1) we first apply a Sparse Autoencoder(SAE) to disentangle the representations into human understandable features. 2) We then present an automatic interpretation framework to interpreted the open-semantic features learned in SAE by the LMMs themselves. We employ this framework to analyze the LLaVA-NeXT-8B model using the LLaVA-OV-72B model, demonstrating that these features can effectively steer the model's behavior. Our results contribute to a deeper understanding of why LMMs excel in specific tasks, including EQ tests, and illuminate the nature of their mistakes along with potential strategies for their rectification. These findings offer new insights into the internal mechanisms of LMMs and suggest parallels with the cognitive processes of the human brain.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 20 of its 41 references that are in the map. First posted on arXiv 22 Nov 2024.

Builds on

20 of 41 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution18 Sep
  2. 2024A Concept-Based Explainability Framework for Large Multimodal Models12 Jun
  3. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks21 Dec
  4. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)29 Sep
  5. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities4 Aug
  6. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models23 Jun
  7. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning11 May

Other references · 13

  1. 2023Visual Instruction Tuningcited by 7,792
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  3. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  4. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  5. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  6. 2022Toy Models of Superpositioncited by 928
  7. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 703
  8. 2020Closed-Form Factorization of Latent Semantics in GANscited by 407
  9. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  10. 2014Object Detectors Emerge in Deep Scene CNNscited by 178
  11. 2024Reducing Hallucinations in Vision-Language Models via Latent Space Steeringcited by 56
  12. 2024Red Teaming GPT-4V: Are GPT-4V Safe Against Uni/Multi-Modal Jailbreak Attacks?cited by 32
1 more
  1. 2013Learning Sparsely Used Overcomplete Dictionaries via Alternating Minimizationcited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 8; 2026: 15.

23 papers.

Sort:

By area:

  1. 2025Sparse Autoencoders Learn Monosemantic Features in Vision-Language Modelscited by 70
  2. 2025SAE-V: Interpreting Multimodal Models for Enhanced Alignmentcited by 22
  3. 2025Visual Representations inside the Language Modelcited by 13
  4. 2025LangBridge: Interpreting Image as a Combination of Language Embeddingscited by 9
  5. 2025From Behavioral Performance to Internal Competence: Interpreting Vision-Language Models with VLM-Lenscited by 9
  6. 2025SUB: Benchmarking CBM Generalization via Synthetic Attribute Substitutionscited by 6
  7. 2025SAVE: Sparse Autoencoder-Driven Visual Information Enhancement for Mitigating Object Hallucinationcited by 3
  8. 2026Inside the Visual Mind: Neuroscience-Motivated Concept Circuits for Interpreting and Steering Vision Transformerscited by 3
  9. 2025Insights into a radiology-specialised multimodal large language model with sparse autoencoderscited by 2
  10. 2026Towards Interpretable Hallucination Analysis and Mitigation in LVLMs via Contrastive Neuron Steeringcited by 2
  11. 2026Improving Sparse Autoencoder with Dynamic Attentioncited by 2
  12. 2026Why Does Reinforcement Learning Generalize? A Feature-Level Mechanistic Study of Post-Training in Large Language Modelscited by 2
11 more
  1. 2026Language Models Can Explain Visual Features via Steeringcited by 1
  2. 2026Beyond Semantics: Disentangling Information Scope in Sparse Autoencoders for CLIPcited by 1
  3. 2026Visual concept ranking uncovers medical shortcuts used by large multimodal modelscited by 0
  4. 2026Dictionary-Aligned Concept Control for Safeguarding Multimodal LLMscited by 0
  5. 2026The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure Modescited by 0
  6. 2026The Hidden Evolution of Disguised Visual Context inside the VLMcited by 0
  7. 2026Same Concept, Different Directions: Cross-Modal Feature Heterogeneity in Sparse Autoencoderscited by 0
  8. 2026When Structured Sparse Autoencoders Learn Consistent Concepts Across Modalitiescited by 0
  9. 2026Latent Cluster Analysis for Vision-Language-Action Modelscited by 0
  10. 2026Taming CoT Obfuscation in VLMs: From Mechanistic Evidence to Activation Enforcementcited by 0
  11. 2026NeuronEye: Query-Guided Visual Concept Activation for Vision-Language Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Human-like object concept representations emerge naturally in multimodal large language models
  2. 2023Beyond Task Performance: Evaluating and Reducing the Flaws of Large Multimodal Models with In-Context Learning
  3. 2025Talking to the brain: Using Large Language Models as Proxies to Model Brain Semantic Representation
  4. 2026Understanding Large Language Models
  5. 2025I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoders
  6. 2025From Black Boxes to Transparent Minds: Evaluating and Enhancing the Theory of Mind in Multimodal Large Language Models
  7. 2023Finding and Editing Multi-Modal Neurons in Pre-Trained Transformers
  8. 2026Mechanistic Insights into Functional Sparsity in Multimodal LLMs via CoRe Heads