# Disentangling Syntax and Semantics in the Brain with Deep Networks

arXiv: https://arxiv.org/abs/2103.01620 · HTML: https://mlsanity.com/paper/2103.01620/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 67 references that are in the map.
- First posted on arXiv 2 Mar 2021.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [Compositionality decomposed: how do neural networks generalise?](https://mlsanity.com/paper/1908.08351/) (2019)
- [GECToR -- Grammatical Error Correction: Tag, Not Rewrite](https://mlsanity.com/paper/2005.12592/) (2020)
- [Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain)](https://mlsanity.com/paper/1905.11833/) (2019)
- [Syntactic Structure from Deep Learning](https://mlsanity.com/paper/2004.10827/) (2020)
- [The emergence of number and syntax units in LSTM language models](https://mlsanity.com/paper/1903.07435/) (2019)
- [Linguistic generalization and compositionality in modern artificial neural networks](https://mlsanity.com/paper/1904.00157/) (2019)
- [Efficient Second-Order TreeCRF for Neural Dependency Parsing](https://mlsanity.com/paper/2005.00975/) (2020)
- [Finding Syntax in Human Encephalography with Beam Search](https://mlsanity.com/paper/1806.04127/) (2018)
- [Brain2Word: Decoding Brain Activity for Language Generation](https://mlsanity.com/paper/2009.04765/) (2020)
- [Relating Simple Sentence Representations in Deep Neural Networks and the Brain](https://mlsanity.com/paper/1906.11861/) (2019)

## Built on by (most cited first, 30 of 40)

- [Toward a realistic model of speech processing in the brain with self-supervised learning](https://mlsanity.com/paper/2206.01685/) (2022)
- [Explaining black box text modules in natural language with language models](https://mlsanity.com/paper/2305.09863/) (2023)
- [Emergent Modularity in Pre-trained Transformers](https://mlsanity.com/paper/2305.18390/) (2023)
- [Contextual Feature Extraction Hierarchies Converge in Large Language Models and the Brain](https://mlsanity.com/paper/2401.17671/) (2024)
- [Neural Language Taskonomy: Which NLP Tasks are the most Predictive of fMRI Brain Activity?](https://mlsanity.com/paper/2205.01404/) (2022)
- [Neural Language Models are not Born Equal to Fit Brain Data, but Training Helps](https://mlsanity.com/paper/2207.03380/) (2022)
- [A polar coordinate system represents syntax in large language models](https://mlsanity.com/paper/2412.05571/) (2024)
- [Decoding Probing: Revealing Internal Linguistic Structures in Neural Language Models using Minimal Pairs](https://mlsanity.com/paper/2403.17299/) (2024)
- [Deep Neural Networks and Brain Alignment: Brain Encoding and Decoding (Survey)](https://mlsanity.com/paper/2307.10246/) (2023)
- [Information-Restricted Neural Language Models Reveal Different Brain Regions' Sensitivity to Semantics, Syntax and Context](https://mlsanity.com/paper/2302.14389/) (2023)
- [Crafting Interpretable Embeddings by Asking LLMs Questions](https://mlsanity.com/paper/2405.16714/) (2024)
- [Long-range and hierarchical language predictions in brains and algorithms](https://mlsanity.com/paper/2111.14232/) (2021)
- [What Are Large Language Models Mapping to in the Brain? A Case Against Over-Reliance on Brain Scores](https://mlsanity.com/paper/2406.01538/) (2024)
- [Coupling Artificial Neurons in BERT and Biological Neurons in the Human Brain](https://mlsanity.com/paper/2303.14871/) (2023)
- [Mapping Brains with Language Models: A Survey](https://mlsanity.com/paper/2306.05126/) (2023)
- [fMRI predictors based on language models of increasing complexity recover brain left lateralization](https://mlsanity.com/paper/2405.17992/) (2024)
- [Differential syntactic and semantic encoding in LLMs](https://mlsanity.com/paper/2601.04765/) (2026)
- [Connecting Neural Response measurements & Computational Models of language: a non-comprehensive guide](https://mlsanity.com/paper/2203.05300/) (2022)
- [Decoupling Content and Expression: Two-Dimensional Detection of AI-Generated Text](https://mlsanity.com/paper/2503.00258/) (2025)
- [Multilevel Interpretability Of Artificial Neural Networks: Leveraging Framework And Methods From Neuroscience](https://mlsanity.com/paper/2408.12664/) (2024)
- [Probing Syntax in Large Language Models: Successes and Remaining Challenges](https://mlsanity.com/paper/2508.03211/) (2025)
- [Computational Models to Study Language Processing in the Human Brain: A Survey](https://mlsanity.com/paper/2403.13368/) (2024)
- [Explanations of Large Language Models Explain Language Representations in the Brain](https://mlsanity.com/paper/2502.14671/) (2025)
- [Deep Learning Models to Study Sentence Comprehension in the Human Brain](https://mlsanity.com/paper/2301.06340/) (2023)
- [Interpretable Next-token Prediction via the Generalized Induction Head](https://mlsanity.com/paper/2411.00066/) (2024)
- [MEG-MASC: a high-quality magneto-encephalography dataset for evaluating natural speech processing](https://mlsanity.com/paper/2208.11488/) (2022)
- [Hierarchical Frequency Tagging Probe (HFTP): A Unified Approach to Investigate Syntactic Structure Representations in Large Language Models and the Human Brain](https://mlsanity.com/paper/2510.13255/) (2025)
- [Emergence and Localisation of Semantic Role Circuits in LLMs](https://mlsanity.com/paper/2511.20910/) (2025)
- [When Language Models Lose Their Mind: The Consequences of Brain Misalignment](https://mlsanity.com/paper/2603.23091/) (2026)
- [Sparse Autoencoders Map Brain-LLM Alignment onto Cortical Semantic Topography](https://mlsanity.com/paper/2605.23035/) (2026)

## Abstract (arXiv)

The activations of language transformers like GPT-2 have been shown to linearly map onto brain activity during speech comprehension. However, the nature of these activations remains largely unknown and presumably conflate distinct linguistic classes. Here, we propose a taxonomy to factorize the high-dimensional activations of language models into four combinatorial classes: lexical, compositional, syntactic, and semantic representations. We then introduce a statistical method to decompose, through the lens of GPT-2's activations, the brain activity of 345 subjects recorded with functional magnetic resonance imaging (fMRI) during the listening of ~4.6 hours of narrated text. The results highlight two findings. First, compositional representations recruit a more widespread cortical network than lexical ones, and encompass the bilateral temporal, parietal and prefrontal cortices. Second, contrary to previous claims, syntax and semantics are not associated with separated modules, but, instead, appear to share a common and distributed neural substrate. Overall, this study introduces a versatile framework to isolate, in the brain activity, the distributed representations of linguistic constructs.