Sparse Autoencoders Map Brain-LLM Alignment onto Cortical Semantic Topography
On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods
also in 1 more
Abstract (arXiv)
Intermediate layers of large language models (LLMs) best predict human brain responses to language, one of the most robust findings in computational neurolinguistics, yet why remains mechanistically unexplained. We address this gap by bridging sparse autoencoders (SAEs) from mechanistic interpretability with neural encoding models, decomposing GPT-2 XL and Llama-3.1-8B into 16K-32K interpretable features per layer. A human-validated taxonomy (kappa ≥ 0.74) reveals that semantic features alone recover 94% of peak encoding performance (r=0.285), substantially exceeding variance-matched baselines (p<0.001, d=1.31). Beyond this aggregate dominance, we test a novel cortical topography prediction: five semantic subcategories derived a priori from three independent neuroscience programs should map onto distinct brain regions. A formal convergence test confirms this alignment (Spearman ρ=0.72, p<0.001; hypergeometric p=0.007), demonstrating that SAE-discovered features recapitulate known cortical semantic organization at a granularity inaccessible to prior methods. SAE features further predict human reading times beyond lexical controls (DeltalogLik=38.4, p<0.001), and an exploratory prediction-error analysis provides preliminary evidence that the brain additionally encodes unexpected semantic content. Results generalize across English, Chinese, and French.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 21 of its 68 references that are in the map. First posted on arXiv 21 May 2026.
Builds on
21 of 68 references.
References · 21
- 2019Similarity of Neural Network Representations Revisitedcited by 1,755
- 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
- 2020Explaining by Removing: A Unified Framework for Model Explanationcited by 218
- 2023Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited by 209
- 2024A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoderscited by 160
- 2019Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain)cited by 128
- 2024Towards Principled Evaluations of Sparse Autoencoders for Interpretability and Controlcited by 86
- 2024RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representationscited by 79
- 2023Testing the Predictions of Surprisal Theory in 11 Languagescited by 77
- 2024Language Models Learn Rare Phenomena from Less Rare Phenomena: The Case of the Missing AANNscited by 65
- 2019Recurrence is required to capture the representational dynamics of the human visual systemcited by 58
- 2024The Geometry of Concepts: Sparse Autoencoder Feature Structurecited by 52
9 more
- 2021Disentangling Syntax and Semantics in the Brain with Deep Networkscited by 40
- 2022Joint processing of linguistic properties in brains and language modelscited by 37
- 2023Dissociating language and thought in large language modelscited by 29
- 2022Neural Language Models are not Born Equal to Fit Brain Data, but Training Helpscited by 26
- 2021The Algonauts Project 2021 Challenge: How the Human Brain Makes Sense of a World in Motioncited by 17
- 2025Neural Brain: A Neuroscience-inspired Framework for Embodied Agentscited by 10
- 2026AudioSAE: Towards Understanding of Audio-Processing Models with Sparse AutoEncoderscited by 8
- 2025BrainExplore: Large-Scale Discovery of Interpretable Visual Representations in the Human Braincited by 5
- 2024Brain-aligning of semantic vectors improves neural decoding of visual stimulicited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Linguistic properties and model scale in brain encoding: from small to compressed language models
- 2025Do Large Language Models Think Like the Brain? Sentence-Level Evidences from Layer-Wise Embeddings and fMRI
- 2026Cross-lingual brain-language model alignment is robust but challenges hierarchical and computational accounts
- 2026Abstraction Induces the Brain Alignment of Language and Speech Models
- 2025Brain-Language Model Alignment: Insights into the Platonic Hypothesis and Intermediate-Layer Advantage
- 2024What Are Large Language Models Mapping to in the Brain? A Case Against Over-Reliance on Brain Scores
- 2026Computational Lesions in Multilingual Language Models Separate Shared and Language-specific Brain Alignment
- 2026Cognitive Expert Language Models Better Align with the Corresponding Brain Systems