# Navigating Brain Language Representations: A Comparative Analysis of Neural Language Models and Psychologically Plausible Models

arXiv: https://arxiv.org/abs/2404.19364 · HTML: https://mlsanity.com/paper/2404.19364/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 42 references that are in the map.
- First posted on arXiv 30 Apr 2024.

## Builds on (most cited first, 10 of 10)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [CogniVal: A Framework for Cognitive Word Embedding Evaluation](https://mlsanity.com/paper/1909.09001/) (2019)
- [Neural Language Taskonomy: Which NLP Tasks are the most Predictive of fMRI Brain Activity?](https://mlsanity.com/paper/2205.01404/) (2022)
- [Neural Language Models are not Born Equal to Fit Brain Data, but Training Helps](https://mlsanity.com/paper/2207.03380/) (2022)
- [Robust Evaluation of Language-Brain Encoding Experiments](https://mlsanity.com/paper/1904.02547/) (2019)
- [MulCogBench: A Multi-modal Cognitive Benchmark Dataset for Evaluating Chinese and English Computational Language Models](https://mlsanity.com/paper/2403.01116/) (2024)
- [Experiential, Distributional and Dependency-based Word Embeddings have Complementary Roles in Decoding Brain Activity](https://mlsanity.com/paper/1711.09285/) (2017)
- [Computational Models to Study Language Processing in the Human Brain: A Survey](https://mlsanity.com/paper/2403.13368/) (2024)
- [Abstract representations of events arise from mental errors in learning and memory](https://mlsanity.com/paper/1805.12491/) (2018)
- [Second-Order Word Embeddings from Nearest Neighbor Topological Features](https://mlsanity.com/paper/1705.08488/) (2017)

## Built on by (most cited first, 4 of 4)

- [Teaching Vision-Language Models to Ask: Resolving Ambiguity in Visual Questions](https://mlsanity.com/paper/2507.13773/) (2025)
- [Improving MLLM's Document Image Machine Translation via Synchronously Self-reviewing Its OCR Proficiency](https://mlsanity.com/paper/2507.08309/) (2025)
- [Discovering Semantic Subdimensions through Disentangled Conceptual Representations](https://mlsanity.com/paper/2508.21436/) (2025)
- [The First ChineseBabyLM Challenge: training data-efficient and cognitively plausible language models for Chinese](https://mlsanity.com/paper/2607.10745/) (2026)

## Abstract (arXiv)

Neural language models, particularly large-scale ones, have been consistently proven to be most effective in predicting brain neural activity across a range of studies. However, previous research overlooked the comparison of these models with psychologically plausible ones. Moreover, evaluations were reliant on limited, single-modality, and English cognitive datasets. To address these questions, we conducted an analysis comparing encoding performance of various neural language models and psychologically plausible models. Our study utilized extensive multi-modal cognitive datasets, examining bilingual word and discourse levels. Surprisingly, our findings revealed that psychologically plausible models outperformed neural language models across diverse contexts, encompassing different modalities such as fMRI and eye-tracking, and spanning languages from English to Chinese. Among psychologically plausible models, the one incorporating embodied information emerged as particularly exceptional. This model demonstrated superior performance at both word and discourse levels, exhibiting robust prediction of brain activation across numerous regions in both English and Chinese.