# Training-Driven Representational Geometry Modularization Predicts Brain Alignment in Language Models

arXiv: https://arxiv.org/abs/2602.07539 · HTML: https://mlsanity.com/paper/2602.07539/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 24 references that are in the map.
- First posted on arXiv 7 Feb 2026.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [Layer by Layer: Uncovering Hidden Representations in Language Models](https://mlsanity.com/paper/2502.02013/) (2025)
- [Large language models implicitly learn to straighten neural sentence trajectories to construct a predictive representation of natural language](https://mlsanity.com/paper/2311.04930/) (2023)
- [From Language to Cognition: How LLMs Outgrow the Human Language Network](https://mlsanity.com/paper/2503.01830/) (2025)
- [Information-Restricted Neural Language Models Reveal Different Brain Regions' Sensitivity to Semantics, Syntax and Context](https://mlsanity.com/paper/2302.14389/) (2023)
- [TopoLM: brain-like spatio-functional organization in a topographic language model](https://mlsanity.com/paper/2410.11516/) (2024)
- [A Brain-like Synergistic Core in LLMs Drives Behaviour and Learning](https://mlsanity.com/paper/2601.06851/) (2026)
- [Far from the Shallow: Brain-Predictive Reasoning Embedding through Residual Disentanglement](https://mlsanity.com/paper/2510.22860/) (2025)
- [Fine-grained Analysis of Brain-LLM Alignment through Input Attribution](https://mlsanity.com/paper/2510.12355/) (2025)

## Abstract (arXiv)

How large language models (LLMs) align with the neural representation and computation of human language is a central question in cognitive science. Using representational geometry as a mechanistic lens, we addressed this by tracking entropy, curvature, and fMRI encoding scores throughout Pythia (70M-1B) training. We identified a geometric modularization where layers self-organize into stable low- and high-complexity clusters. The low-complexity module, characterized by reduced entropy and curvature, consistently better predicted human language network activity. This alignment followed heterogeneous spatial-temporal trajectories: rapid and stable in temporal regions (AntTemp, PostTemp), but delayed and dynamic in frontal areas (IFG, IFGorb). Crucially, reduced curvature remained a robust predictor of model-brain alignment even after controlling for training progress, an effect that strengthened with model scale. These results links training-driven geometric reorganization to temporal-frontal functional specialization, suggesting that representational smoothing facilitates neural-like linguistic processing.