Do Models See in Line with Human Vision? Probing the Correspondence Between LVLM Representations and EEG Signals
On the map: LLM reasoning → Multimodal reasoning
also in 1 more
Abstract (arXiv)
Large Vision Language Models (LVLMs) exhibit strong visual understanding and reasoning abilities. However, whether their internal representations reflect human visual cognition is still under-explored. In this paper, we address this by quantifying LVLM-brain alignment using image-evoked Electroencephalogram (EEG) signals, analyzing the effects of model architecture, scale, and image type. Specifically, by using ridge regression and representational similarity analysis, we compare visual representations from 32 open-source LVLMs with corresponding EEG responses. We observe a structured LVLM-brain correspondence: First, intermediate layers (8-16) show peak alignment with EEG activity in the 100-300 ms window, consistent with hierarchical human visual processing. Secondly, multimodal architectural design contributes 3.4 more to brain alignment than parameter scaling, and models with stronger downstream visual performance exhibit higher EEG similarity. Thirdly, spatiotemporal patterns further align with known cortical visual pathways. These results demonstrate that LVLMs learn human-aligned visual representations and establish neural alignment as a biologically grounded benchmark for evaluating and improving LVLMs. In addition, those results could provide insights that may inform the development of neuro-inspired applications.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. First posted on arXiv 9 Mar 2026.
Builds on
No reference list yet.
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Vision-Language Models Align with Human Neural Representations in Concept Processing
- 2025Correlating instruction-tuning (in multimodal models) with vision-language processing (in the brain)
- 2026Do VLMs Align Better with Humans than LLMs during Natural Reading?
- 2026Visualizing the Invisible: Generative Visual Grounding Empowers Universal EEG Understanding in MLLMs
- 2025Uncovering Brain-Like Hierarchical Patterns in Vision-Language Models through fMRI-Based Neural Encoding
- 2026Brain alignment of reasoning and action representations from vision-language and action models during naturalistic gameplay
- 2025Visual Language Models show widespread visual deficits on neuropsychological tests
- 2025Task-conditioned probing of instruction-tuned multimodal LLMs: Region-specific brain alignment patterns under naturalistic stimuli