# Brain Score Tracks Shared Properties of Languages: Evidence from Many Natural Languages and Structured Sequences

arXiv: https://arxiv.org/abs/2604.15503 · HTML: https://mlsanity.com/paper/2604.15503/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 24 references that are in the map.
- First posted on arXiv 16 Apr 2026.

## Builds on (most cited first, 10 of 10)

- [The Stack: 3 TB of permissively licensed source code](https://mlsanity.com/paper/2211.15533/) (2022)
- [Between Circuits and Chomsky: Pre-pretraining on Formal Languages Imparts Linguistic Biases](https://mlsanity.com/paper/2502.19249/) (2025)
- [Injecting structural hints: Using language models to study inductive biases in language learning](https://mlsanity.com/paper/2304.13060/) (2023)
- [Do large language models resemble humans in language use?](https://mlsanity.com/paper/2303.08014/) (2023)
- [Code Pretraining Improves Entity Tracking Abilities of Language Models](https://mlsanity.com/paper/2405.21068/) (2024)
- [Pretraining with Artificial Language: Studying Transferable Knowledge in Language Models](https://mlsanity.com/paper/2203.10326/) (2022)
- [Neural Language Models are not Born Equal to Fit Brain Data, but Training Helps](https://mlsanity.com/paper/2207.03380/) (2022)
- [On the Transferability of Pre-trained Language Models: A Study from Artificial Datasets](https://mlsanity.com/paper/2109.03537/) (2021)
- [What Are Large Language Models Mapping to in the Brain? A Case Against Over-Reliance on Brain Scores](https://mlsanity.com/paper/2406.01538/) (2024)
- [Procedural Pretraining: Warming Up Language Models with Abstract Data](https://mlsanity.com/paper/2601.21725/) (2026)

## Abstract (arXiv)

Recent breakthroughs in language models (LMs) using neural networks have raised the question: how similar are these models' processing to human language processing? Results using a framework called Brain Score (BS) -- predicting fMRI activations during reading from LM activations -- have been used to argue for a high degree of similarity. To understand this similarity, we conduct experiments by training LMs on various types of input data and evaluate them on BS. We find that models trained on various natural languages from many different language families have very similar BS performance. LMs trained on other structured data -- the human genome, Python, and pure hierarchical structure (nested parentheses) -- also perform reasonably well and close to natural languages in some cases. These findings suggest that BS can highlight language models' ability to extract common structure across natural languages, but that the metric may not be sensitive enough to allow us to infer human-like processing from a high BS score alone.