# Modular Cognitive Architecture Emerges in Large Language Models

arXiv: https://arxiv.org/abs/2608.13567 · HTML: https://mlsanity.com/paper/2608.13567/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 57 references that are in the map.
- First posted on arXiv 27 Jun 2026.

## Builds on (most cited first, 23 of 23)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Knowledge Neurons in Pretrained Transformers](https://mlsanity.com/paper/2104.08696/) (2021)
- [BLiMP: The Benchmark of Linguistic Minimal Pairs for English](https://mlsanity.com/paper/1912.00582/) (2019)
- [2 OLMo 2 Furious](https://mlsanity.com/paper/2501.00656/) (2024)
- [Attribution Patching Outperforms Automated Circuit Discovery](https://mlsanity.com/paper/2310.10348/) (2023)
- [How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model](https://mlsanity.com/paper/2305.00586/) (2023)
- [Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Tracking](https://mlsanity.com/paper/2402.14811/) (2024)
- [Learning to Generate Reviews and Discovering Sentiment](https://mlsanity.com/paper/1704.01444/) (2017)
- [REASONING GYM: Reasoning Environments for Reinforcement Learning with Verifiable Rewards](https://mlsanity.com/paper/2505.24760/) (2025)
- [Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Masks](https://mlsanity.com/paper/2010.02066/) (2020)
- [The emergence of number and syntax units in LSTM language models](https://mlsanity.com/paper/1903.07435/) (2019)
- [A Mechanistic Interpretation of Arithmetic Reasoning in Language Models using Causal Mediation Analysis](https://mlsanity.com/paper/2305.15054/) (2023)
- [PROST: Physical Reasoning of Objects through Space and Time](https://mlsanity.com/paper/2106.03634/) (2021)
- [Large Language Models Pass the Turing Test](https://mlsanity.com/paper/2503.23674/) (2025)
- [NEWTON: Are Large Language Models Capable of Physical Reasoning?](https://mlsanity.com/paper/2310.07018/) (2023)
- [Are formal and functional linguistic mechanisms dissociated in language models?](https://mlsanity.com/paper/2503.11302/) (2025)
- [Different types of syntactic agreement recruit the same units within large language models](https://mlsanity.com/paper/2512.03676/) (2025)
- [Discovering Functionally Selective Brain Regions with a Deep Topographic Multimodal Model](https://mlsanity.com/paper/2606.09770/) (2026)
- [Discovering Decoupled Functional Modules in Large Language Models](https://mlsanity.com/paper/2603.17823/) (2026)

## Built on by (most cited first, 3 of 3)

- [Pre-carved Niches: The Formation Dynamics of Modular Task Partitions in Early LLM Training](https://mlsanity.com/paper/2609.01170/) (2026)
- [Logical subspace in LLMs](https://mlsanity.com/paper/2609.32907/) (2026)
- [Task Context Shapes Component Roles in Language Models](https://mlsanity.com/paper/2610.13895/) (2026)

## Abstract (arXiv)

The human brain exhibits a striking degree of functional specialization, with distinct networks supporting language, formal reasoning, reasoning about other minds, and reasoning about the physical world. Is this modular organization a fundamental principle of how intelligent systems must be built, or an evolutionary accident specific to biological brains? Here, we test whether a similar organization emerges in Large Language Models--another class of intelligent systems created through a very different optimization process. Using circuit analyses across N=46 tasks spanning four cognitive domains (language, formal reasoning, social reasoning, physical reasoning), we find that LLMs develop a modular architecture that mirrors the human brain: tasks drawing on the same network in humans recruit overlapping neurons in LLMs, whereas tasks drawing on different networks recruit distinct neurons. The convergent emergence of modularity in brains and neural networks suggests that it may be a fundamental property of intelligent systems.