# Cross-modal Information Flow in Multimodal Large Language Models

arXiv: https://arxiv.org/abs/2411.18620 · HTML: https://mlsanity.com/paper/2411.18620/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 36 of its 49 references that are in the map.
- First posted on arXiv 27 Nov 2024.

## Placed in

- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 30 of 36)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints](https://mlsanity.com/paper/2305.13245/) (2023)
- [Transformer Feed-Forward Layers Are Key-Value Memories](https://mlsanity.com/paper/2012.14913/) (2020)
- [OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge](https://mlsanity.com/paper/1906.00067/) (2019)
- [Progress measures for grokking via mechanistic interpretability](https://mlsanity.com/paper/2301.05217/) (2023)
- [A-OKVQA: A Benchmark for Visual Question Answering using World Knowledge](https://mlsanity.com/paper/2206.01718/) (2022)
- [EVA: Exploring the Limits of Masked Visual Representation Learning at Scale](https://mlsanity.com/paper/2211.07636/) (2022)
- [Dissecting Recall of Factual Associations in Auto-Regressive Language Models](https://mlsanity.com/paper/2304.14767/) (2023)
- [Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models](https://mlsanity.com/paper/2403.18814/) (2024)
- [Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers](https://mlsanity.com/paper/2103.15679/) (2021)
- [Analyzing Transformers in Embedding Space](https://mlsanity.com/paper/2209.02535/) (2022)
- [Towards Interpreting Visual Information Processing in Vision-Language Models](https://mlsanity.com/paper/2410.07149/) (2024)
- [Probing Image-Language Transformers for Verb Understanding](https://mlsanity.com/paper/2106.09141/) (2021)
- [Behind the Scene: Revealing the Secrets of Pre-trained Vision-and-Language Models](https://mlsanity.com/paper/2005.07310/) (2020)
- [Vision-and-Language or Vision-for-Language? On Cross-Modal Influence in Multimodal Transformers](https://mlsanity.com/paper/2109.04448/) (2021)
- [Towards Vision-Language Mechanistic Interpretability: A Causal Tracing Tool for BLIP](https://mlsanity.com/paper/2308.14179/) (2023)
- [VL-InterpreT: An Interactive Visualization Tool for Interpreting Vision-Language Transformers](https://mlsanity.com/paper/2203.17247/) (2022)

## Built on by (most cited first, 30 of 67)

- [Visual Representation Alignment for Multimodal Large Language Models](https://mlsanity.com/paper/2509.07979/) (2025)
- [Same Task, Different Circuits: Disentangling Modality-Specific Mechanisms in VLMs](https://mlsanity.com/paper/2506.09047/) (2025)
- [HiddenDetect: Detecting Jailbreak Attacks against Large Vision-Language Models via Monitoring Hidden States](https://mlsanity.com/paper/2502.14744/) (2025)
- [Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models](https://mlsanity.com/paper/2601.14004/) (2026)
- [VFlowOpt: A Token Pruning Framework for LMMs with Visual Information Flow-Guided Optimization](https://mlsanity.com/paper/2508.05211/) (2025)
- [VisiPruner: Decoding Discontinuous Cross-Modal Dynamics for Efficient Multimodal LLMs](https://mlsanity.com/paper/2510.17205/) (2025)
- [Growing a Multi-head Twig via Distillation and Reinforcement Learning to Accelerate Large Vision-Language Models](https://mlsanity.com/paper/2503.14075/) (2025)
- [LatentLens: Revealing Highly Interpretable Visual Tokens in LLMs](https://mlsanity.com/paper/2602.00462/) (2026)
- [N"uwa: Mending the Spatial Integrity Torn by VLM Token Pruning](https://mlsanity.com/paper/2602.02951/) (2026)
- [Visual Multi-Agent System: Mitigating Hallucination Snowballing via Visual Flow](https://mlsanity.com/paper/2509.21789/) (2025)
- [HiconAgent: History Context-aware Policy Optimization for GUI Agents](https://mlsanity.com/paper/2512.01763/) (2025)
- [IVC-Prune: Revealing the Implicit Visual Coordinates in LVLMs for Vision Token Pruning](https://mlsanity.com/paper/2602.03060/) (2026)
- [LoRASculpt: Sculpting LoRA for Harmonizing General and Specialized Knowledge in Multimodal Large Language Models](https://mlsanity.com/paper/2503.16843/) (2025)
- [Map the Flow: Revealing Hidden Pathways of Information in VideoLLMs](https://mlsanity.com/paper/2510.13251/) (2025)
- [Scone: Bridging Composition and Distinction in Subject-Driven Image Generation via Unified Understanding-Generation Modeling](https://mlsanity.com/paper/2512.12675/) (2025)
- [ViCA: Efficient Multimodal LLMs with Vision-Only Cross-Attention](https://mlsanity.com/paper/2602.07574/) (2026)
- [AdaRD-key: Adaptive Relevance-Diversity Keyframe Sampling for Long-form Video understanding](https://mlsanity.com/paper/2510.02778/) (2025)
- [Tell Model Where to Look: Mitigating Hallucinations in MLLMs by Vision-Guided Attention](https://mlsanity.com/paper/2511.20032/) (2025)
- [Do Audio-Visual Large Language Models Really See and Hear?](https://mlsanity.com/paper/2604.02605/) (2026)
- [TARAC: Mitigating Hallucination in LVLMs via Temporal Attention Real-time Accumulative Connection](https://mlsanity.com/paper/2504.04099/) (2025)
- [FlowMM: Cross-Modal Information Flow Guided KV Cache Merging for Efficient Multimodal Context Inference](https://mlsanity.com/paper/2511.05534/) (2025)
- [Interpretable Debiasing of Vision-Language Models for Social Fairness](https://mlsanity.com/paper/2602.24014/) (2026)
- [MMA-ASIA: A Multilingual and Multimodal Alignment Framework for Culturally-Grounded Evaluation](https://mlsanity.com/paper/2510.08608/) (2025)
- [Seeing Beyond Words: Self-Supervised Visual Learning for Multimodal Large Language Models](https://mlsanity.com/paper/2512.15885/) (2025)
- [LinkedOut: Linking World Knowledge Representation Out of Video LLM for Next-Generation Video Recommendation](https://mlsanity.com/paper/2512.16891/) (2025)
- [Test-Time Attention Purification for Backdoored Large Vision Language Models](https://mlsanity.com/paper/2603.12989/) (2026)
- [Pathways of Visual Information Flow in Vision-Language Models](https://mlsanity.com/paper/2607.03358/) (2026)
- [TimeViper: A Hybrid Mamba-Transformer Vision-Language Model for Efficient Long Video Understanding](https://mlsanity.com/paper/2511.16595/) (2025)
- [A Comprehensive Information-Decomposition Analysis of Large Vision-Language Models](https://mlsanity.com/paper/2603.29676/) (2026)
- [ThanoRA: Task Heterogeneity-Aware Multi-Task Low-Rank Adaptation](https://mlsanity.com/paper/2505.18640/) (2025)

## Abstract (arXiv)

The recent advancements in auto-regressive multimodal large language models (MLLMs) have demonstrated promising progress for vision-language tasks. While there exists a variety of studies investigating the processing of linguistic information within large language models, little is currently known about the inner working mechanism of MLLMs and how linguistic and visual information interact within these models. In this study, we aim to fill this gap by examining the information flow between different modalities -- language and vision -- in MLLMs, focusing on visual question answering. Specifically, given an image-question pair as input, we investigate where in the model and how the visual and linguistic information are combined to generate the final prediction. Conducting experiments with a series of models from the LLaVA series, we find that there are two distinct stages in the process of integration of the two modalities. In the lower layers, the model first transfers the more general visual features of the whole image into the representations of (linguistic) question tokens. In the middle layers, it once again transfers visual information about specific objects relevant to the question to the respective token positions of the question. Finally, in the higher layers, the resulting multimodal representation is propagated to the last position of the input sequence for the final prediction. Overall, our findings provide a new and comprehensive perspective on the spatial and functional aspects of image and language processing in the MLLMs, thereby facilitating future research into multimodal information localization and editing. Our code and collected dataset are released here: https://github.com/FightingFighting/cross-modal-information-flow-in-MLLM.git.