# Mapping Brains with Language Models: A Survey

arXiv: https://arxiv.org/abs/2306.05126 · HTML: https://mlsanity.com/paper/2306.05126/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 80 references that are in the map.
- First posted on arXiv 8 Jun 2023.

## Builds on (most cited first, 26 of 26)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [BookSum: A Collection of Datasets for Long-form Narrative Summarization](https://mlsanity.com/paper/2105.08209/) (2021)
- [Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain)](https://mlsanity.com/paper/1905.11833/) (2019)
- [Toward a realistic model of speech processing in the brain with self-supervised learning](https://mlsanity.com/paper/2206.01685/) (2022)
- [Disentangling Syntax and Semantics in the Brain with Deep Networks](https://mlsanity.com/paper/2103.01620/) (2021)
- [Blackbox meets blackbox: Representational Similarity and Stability Analysis of Neural Language Models and Brains](https://mlsanity.com/paper/1906.01539/) (2019)
- [Inducing brain-relevant bias in natural language processing models](https://mlsanity.com/paper/1911.03268/) (2019)
- [Linking artificial and human neural representations of language](https://mlsanity.com/paper/1910.01244/) (2019)
- [Self-supervised models of audio effectively explain human cortical responses to speech](https://mlsanity.com/paper/2205.14252/) (2022)
- [Joint processing of linguistic properties in brains and language models](https://mlsanity.com/paper/2212.08094/) (2022)
- [Low-Dimensional Structure in the Space of Language Representations is Reflected in Brain Responses](https://mlsanity.com/paper/2106.05426/) (2021)
- [CogniVal: A Framework for Cognitive Word Embedding Evaluation](https://mlsanity.com/paper/1909.09001/) (2019)
- [Neural Language Taskonomy: Which NLP Tasks are the most Predictive of fMRI Brain Activity?](https://mlsanity.com/paper/2205.01404/) (2022)
- [Neural Language Models are not Born Equal to Fit Brain Data, but Training Helps](https://mlsanity.com/paper/2207.03380/) (2022)
- [Training language models to summarize narratives improves brain alignment](https://mlsanity.com/paper/2212.10898/) (2022)
- [Interpreting Encoding and Decoding Models](https://mlsanity.com/paper/1812.00278/) (2018)
- [Language models and brains align due to more than next-word prediction and word-level information](https://mlsanity.com/paper/2212.00596/) (2022)
- [Does injecting linguistic structure into language models lead to better alignment with brain recordings?](https://mlsanity.com/paper/2101.12608/) (2021)
- [Long-range and hierarchical language predictions in brains and algorithms](https://mlsanity.com/paper/2111.14232/) (2021)
- [Same Cause; Different Effects in the Brain](https://mlsanity.com/paper/2202.10376/) (2022)
- [Beyond linear regression: mapping models in cognitive neuroscience should align with research goals](https://mlsanity.com/paper/2208.10668/) (2022)
- [Does the brain represent words? An evaluation of brain decoding studies of language understanding](https://mlsanity.com/paper/1806.00591/) (2018)
- [Robust Evaluation of Language-Brain Encoding Experiments](https://mlsanity.com/paper/1904.02547/) (2019)
- [fMRI Semantic Category Decoding using Linguistic Encoding of Word Embeddings](https://mlsanity.com/paper/1806.05177/) (2018)
- [Bridging LSTM Architecture and the Neural Dynamics during Reading](https://mlsanity.com/paper/1604.06635/) (2016)

## Built on by (most cited first, 8 of 8)

- [Deep Neural Networks and Brain Alignment: Brain Encoding and Decoding (Survey)](https://mlsanity.com/paper/2307.10246/) (2023)
- [Improving Semantic Understanding in Speech Language Models via Brain-tuning](https://mlsanity.com/paper/2410.09230/) (2024)
- [Brain-Language Model Alignment: Insights into the Platonic Hypothesis and Intermediate-Layer Advantage](https://mlsanity.com/paper/2510.17833/) (2025)
- [Language models align with brain regions that represent concepts across modalities](https://mlsanity.com/paper/2508.11536/) (2025)
- [Modality-Agnostic fMRI Decoding of Vision and Language](https://mlsanity.com/paper/2403.11771/) (2024)
- [Integrating Cognitive Processing Signals into Language Models: A Review of Advances, Applications and Future Directions](https://mlsanity.com/paper/2504.06843/) (2025)
- [Can "consciousness" be observed from large language model (LLM) internal states? Dissecting LLM representations obtained from Theory of Mind test with Integrated Information Theory and Span Representation analysis](https://mlsanity.com/paper/2506.22516/) (2025)
- [On the Shape of Brainscores for Large Language Models (LLMs)](https://mlsanity.com/paper/2405.06725/) (2024)

## Abstract (arXiv)

Over the years, many researchers have seemingly made the same observation: Brain and language model activations exhibit some structural similarities, enabling linear partial mappings between features extracted from neural recordings and computational language models. In an attempt to evaluate how much evidence has been accumulated for this observation, we survey over 30 studies spanning 10 datasets and 8 metrics. How much evidence has been accumulated, and what, if anything, is missing before we can draw conclusions? Our analysis of the evaluation methods used in the literature reveals that some of the metrics are less conservative. We also find that the accumulated evidence, for now, remains ambiguous, but correlations with model size and quality provide grounds for cautious optimism.