# Cross-Modal Projection in Multimodal LLMs Doesn't Really Project Visual Attributes to Textual Space

arXiv: https://arxiv.org/abs/2402.16832 · HTML: https://mlsanity.com/paper/2402.16832/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 18 of its 35 references that are in the map.
- First posted on arXiv 26 Feb 2024.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM training and visual instruction tuning](https://mlsanity.com/mllm/mllm-training-and-visual-instruction-tuning/)

## Builds on (most cited first, 18 of 18)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [Video-LLaVA: Learning United Visual Representation by Alignment Before Projection](https://mlsanity.com/paper/2311.10122/) (2023)
- [LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day](https://mlsanity.com/paper/2306.00890/) (2023)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [Learning Representations by Maximizing Mutual Information Across Views](https://mlsanity.com/paper/1906.00910/) (2019)
- ["Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters](https://mlsanity.com/paper/2310.09219/) (2023)
- [AnyMAL: An Efficient and Scalable Any-Modality Augmented Language Model](https://mlsanity.com/paper/2309.16058/) (2023)
- [CrisisMMD: Multimodal Twitter Datasets from Natural Disasters](https://mlsanity.com/paper/1805.00713/) (2018)
- [Multimodal Neurons in Pretrained Text-Only Transformers](https://mlsanity.com/paper/2308.01544/) (2023)
- [Finding and Editing Multi-Modal Neurons in Pre-Trained Transformers](https://mlsanity.com/paper/2311.07470/) (2023)
- [Analysis of Social Media Data using Multimodal Deep Learning for Disaster Response](https://mlsanity.com/paper/2004.11838/) (2020)

## Built on by (most cited first, 22 of 22)

- [CALF: Aligning LLMs for Time Series Forecasting via Cross-modal Fine-Tuning](https://mlsanity.com/paper/2403.07300/) (2024)
- [Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2412.02104/) (2024)
- [Visual Representation Alignment for Multimodal Large Language Models](https://mlsanity.com/paper/2509.07979/) (2025)
- [Visual Large Language Models for Generalized and Specialized Applications](https://mlsanity.com/paper/2501.02765/) (2025)
- [Learning to See Before Seeing: Demystifying LLM Visual Priors from Language Pre-training](https://mlsanity.com/paper/2509.26625/) (2025)
- [LUMIA: Linear probing for Unimodal and MultiModal Membership Inference Attacks leveraging internal LLM states](https://mlsanity.com/paper/2411.19876/) (2024)
- [Turning Internal Gap into Self-Improvement: Promoting the Generation-Understanding Unification in MLLMs](https://mlsanity.com/paper/2507.16663/) (2025)
- [Merge then Realign: Simple and Effective Modality-Incremental Continual Learning for Multimodal LLMs](https://mlsanity.com/paper/2503.07663/) (2025)
- [Do MLLMs Really See It: Reinforcing Visual Attention in Multimodal LLMs](https://mlsanity.com/paper/2602.08241/) (2026)
- [LLMs Can Compensate for Deficiencies in Visual Representations](https://mlsanity.com/paper/2506.05439/) (2025)
- [MonoSR: Open-Vocabulary Spatial Reasoning from Monocular Images](https://mlsanity.com/paper/2511.19119/) (2025)
- [Phase Diagram of Vision Large Language Models Inference: A Perspective from Interaction across Image and Instruction](https://mlsanity.com/paper/2411.00646/) (2024)
- [A Framework for Situating Innovations, Opportunities, and Challenges in Advancing Vertical Systems with Large AI Models](https://mlsanity.com/paper/2504.02793/) (2025)
- [A Survey on Interpretability in Visual Recognition](https://mlsanity.com/paper/2507.11099/) (2025)
- [Intriguing Properties of Large Language and Vision Models](https://mlsanity.com/paper/2410.04751/) (2024)
- [Lost in the Hype: Revealing and Dissecting the Performance Degradation of Medical Multimodal Large Language Models in Image Classification](https://mlsanity.com/paper/2604.08333/) (2026)
- [CheXmix: Unified Generative Pretraining for Vision Language Models in Medical Imaging](https://mlsanity.com/paper/2604.22989/) (2026)
- [SSDA: Bridging Spectral and Structural Gaps via Dual Adaptation for Vision-Based Time Series Forecasting](https://mlsanity.com/paper/2605.12550/) (2026)
- [CircuitProbe: Tracing Visual Temporal Evidence Flow in Video Language Models](https://mlsanity.com/paper/2507.19420/) (2025)
- [LLM-based Fusion of Multi-modal Features for Commercial Memorability Prediction](https://mlsanity.com/paper/2510.22829/) (2025)
- [Abstracting Cross-Domain Action Sequences into Interpretable Workflows](https://mlsanity.com/paper/2606.14654/) (2026)
- [Rethinking Projector Training in Multimodal LLMs](https://mlsanity.com/paper/2610.18598/) (2026)

## Abstract (arXiv)

Multimodal large language models (MLLMs) like LLaVA and GPT-4(V) enable general-purpose conversations about images with the language modality. As off-the-shelf MLLMs may have limited capabilities on images from domains like dermatology and agriculture, they must be fine-tuned to unlock domain-specific applications. The prevalent architecture of current open-source MLLMs comprises two major modules: an image-language (cross-modal) projection network and a large language model. It is desirable to understand the roles of these two modules in modeling domain-specific visual attributes to inform the design of future models and streamline the interpretability efforts on the current models. To this end, via experiments on 4 datasets and under 2 fine-tuning settings, we find that as the MLLM is fine-tuned, it indeed gains domain-specific visual capabilities, but the updates do not lead to the projection extracting relevant domain-specific visual attributes. Our results indicate that the domain-specific visual attributes are modeled by the LLM, even when only the projection is fine-tuned. Through this study, we offer a potential reinterpretation of the role of cross-modal projections in MLLM architectures. Project webpage: https://claws-lab.github.io/projection-in-MLLMs/