On Representational Dissociation of Language and Arithmetic in Large Language Models
Not on the map. Why?
Abstract (arXiv)
The association between language and (non-linguistic) thinking ability in humans has long been debated, and recently, neuroscientific evidence of brain activity patterns has been considered. Such a scientific context naturally raises an interdisciplinary question -- what about such a language-thought dissociation in large language models (LLMs)? In this paper, as an initial foray, we explore this question by focusing on simple arithmetic skills (e.g., 1+2= ?) as a thinking ability and analyzing the geometry of their encoding in LLMs' representation space. Our experiments with linear classifiers and cluster separability tests demonstrate that simple arithmetic equations and general language input are encoded in completely separated regions in LLMs' internal representation space across all the layers, which is also supported with more controlled stimuli (e.g., spelled-out equations). These tentatively suggest that arithmetic reasoning is mapped into a distinct region from general language input, which is in line with the neuroscientific observations of human brain activations, while we also point out their somewhat cognitively implausible geometric properties.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 16 of its 35 references that are in the map. First posted on arXiv 17 Feb 2025.
Builds on
16 of 35 references.
References · 16
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2016Understanding intermediate layers using linear classifier probescited by 1,279
- 2020Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuningcited by 658
- 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 436
- 2023MADLAD-400: A Multilingual And Document-Level Large Audited Datasetcited by 180
- 2021BERT Busters: Outlier Dimensions that Disrupt Transformerscited by 110
- 2022What Artificial Neural Networks Can Tell Us About Human Language Acquisitioncited by 97
- 2024On the Multilingual Ability of Decoder-based Pre-trained Language Models: Finding and Controlling Language-Specific Neuronscited by 83
- 2024The Semantic Hub Hypothesis: Language Models Share Semantic Representations Across Languages and Modalitiescited by 69
- 2022Outliers Dimensions that Disrupt Transformers Are Driven by Frequencycited by 55
- 2023Instruction-tuning Aligns LLMs to the Human Braincited by 39
4 more
- 2018How deep is deep enough? -- Quantifying class separability in the hidden layers of deep neural networkscited by 28
- 2022Naturalistic Causal Probing for Morpho-Syntaxcited by 19
- 2024Language Models Encode the Value of Numbers Linearlycited by 19
- 2024Interpretability of Language Models via Task Spacescited by 4
Built on by
Citing papers per year: 2025: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Disentangling Mathematical Reasoning in LLMs: A Methodological Investigation of Internal Mechanisms
- 2026Abstract representational geometry supports inference in large language models
- 2024The neural correlates of logical-mathematical symbol systems processing resemble that of spatial cognition more than natural language processing
- 2024An Investigation of Neuron Activation as a Unified Lens to Explain Chain-of-Thought Eliciting Arithmetic Reasoning of LLMs
- 2026NeuReasoner: Theory-grounded Mapping of Reasoning Elicitation Boundaries
- 2025Large Language Models Show Signs of Alignment with Human Neurocognition During Abstract Reasoning
- 2023A Mechanistic Interpretation of Arithmetic Reasoning in Language Models using Causal Mediation Analysis
- 2026Beyond representational alignment with brain-guided language models for robust reasoning