# Scaling and context steer LLMs along the same computational path as the human brain

arXiv: https://arxiv.org/abs/2512.01591 · HTML: https://mlsanity.com/paper/2512.01591/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 48 references that are in the map.
- First posted on arXiv 1 Dec 2025.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [KAN: Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2404.19756/) (2024)
- [The Platonic Representation Hypothesis](https://mlsanity.com/paper/2405.07987/) (2024)
- [Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain)](https://mlsanity.com/paper/1905.11833/) (2019)
- [Cognitive Science in the era of Artificial Intelligence: A roadmap for reverse-engineering the infant language-learner](https://mlsanity.com/paper/1607.08723/) (2016)
- [Scaling laws for language encoding models in fMRI](https://mlsanity.com/paper/2305.11863/) (2023)
- [Toward a realistic model of speech processing in the brain with self-supervised learning](https://mlsanity.com/paper/2206.01685/) (2022)
- [Self-supervised models of audio effectively explain human cortical responses to speech](https://mlsanity.com/paper/2205.14252/) (2022)
- [Fully Connected Deep Structured Networks](https://mlsanity.com/paper/1503.02351/) (2015)
- [Scaling laws for decoding images from brain activity](https://mlsanity.com/paper/2501.15322/) (2025)
- [Decoding individual words from non-invasive brain recordings across 723 participants](https://mlsanity.com/paper/2412.17829/) (2024)
- [Controlled Evaluation of Syntactic Knowledge in Multilingual Language Models](https://mlsanity.com/paper/2411.07474/) (2024)
- [fMRI predictors based on language models of increasing complexity recover brain left lateralization](https://mlsanity.com/paper/2405.17992/) (2024)
- [Towards Data-Efficient Language Models: A Child-Inspired Approach to Language Learning](https://mlsanity.com/paper/2503.04611/) (2025)
- [The Temporal Structure of Language Processing in the Human Brain Corresponds to The Layered Hierarchy of Deep Language Models](https://mlsanity.com/paper/2310.07106/) (2023)

## Built on by (most cited first, 1 of 1)

- [Brain-Conditioned Action Policies for Neural Motor Decoding](https://mlsanity.com/paper/2609.34561/) (2026)

## Abstract (arXiv)

Recent studies suggest that the representations learned by large language models (LLMs) are partially aligned to those of the human brain. However, whether and why this alignment score arises from a similar sequence of computations remains elusive. In this study, we explore this question by examining temporally-resolved brain signals of participants listening to 10 hours of an audiobook. We study these neural dynamics jointly with a benchmark encompassing 22 LLMs varying in size and architecture type. Our analyses confirm that LLMs and the brain generate representations in a similar order: specifically, activations in the initial layers of LLMs tend to best align with early brain responses, while the deeper layers of LLMs tend to best align with later brain responses. This brain-LLM alignment is consistent across transformers and recurrent architectures. However, its emergence depends on both model size and context length. Overall, this study sheds light on the sequential nature of computations and the factors underlying the partial convergence between biological and artificial neural networks.