mlsanityData to 7 Oct 2026

Training-Driven Representational Geometry Modularization Predicts Brain Alignment in Language Models

arXiv 2602.07539 ↗First posted 7 Feb 2026q-bio.NC cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

How large language models (LLMs) align with the neural representation and computation of human language is a central question in cognitive science. Using representational geometry as a mechanistic lens, we addressed this by tracking entropy, curvature, and fMRI encoding scores throughout Pythia (70M-1B) training. We identified a geometric modularization where layers self-organize into stable low- and high-complexity clusters. The low-complexity module, characterized by reduced entropy and curvature, consistently better predicted human language network activity. This alignment followed heterogeneous spatial-temporal trajectories: rapid and stable in temporal regions (AntTemp, PostTemp), but delayed and dynamic in frontal areas (IFG, IFGorb). Crucially, reduced curvature remained a robust predictor of model-brain alignment even after controlling for training progress, an effect that strengthened with model scale. These results links training-driven geometric reorganization to temporal-frontal functional specialization, suggesting that representational smoothing facilitates neural-like linguistic processing.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 11 of its 24 references that are in the map. First posted on arXiv 7 Feb 2026.

Builds on

11 of 24 references.

References · 11

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  3. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  4. 2025Layer by Layer: Uncovering Hidden Representations in Language Modelscited by 293
  5. 2023Large language models implicitly learn to straighten neural sentence trajectories to construct a predictive representation of natural languagecited by 32
  6. 2025From Language to Cognition: How LLMs Outgrow the Human Language Networkcited by 30
  7. 2023Information-Restricted Neural Language Models Reveal Different Brain Regions' Sensitivity to Semantics, Syntax and Contextcited by 13
  8. 2024TopoLM: brain-like spatio-functional organization in a topographic language modelcited by 12
  9. 2025Far from the Shallow: Brain-Predictive Reasoning Embedding through Residual Disentanglementcited by 4
  10. 2026A Brain-like Synergistic Core in LLMs Drives Behaviour and Learningcited by 4
  11. 2025Fine-grained Analysis of Brain-LLM Alignment through Input Attributioncited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Linguistic properties and model scale in brain encoding: from small to compressed language models
  2. 2025Brain-Language Model Alignment: Insights into the Platonic Hypothesis and Intermediate-Layer Advantage
  3. 2026Cross-lingual brain-language model alignment is robust but challenges hierarchical and computational accounts
  4. 2026Abstract representational geometry supports inference in large language models
  5. 2026Sparse Autoencoders Map Brain-LLM Alignment onto Cortical Semantic Topography
  6. 2026The Platonic brain bridge hypothesis: human brain networks as an architectural prior for multimodal large language models
  7. 2025Do Large Language Models Think Like the Brain? Sentence-Level Evidences from Layer-Wise Embeddings and fMRI
  8. 2026Abstraction Induces the Brain Alignment of Language and Speech Models