# The Representational Geometry of Number

arXiv: https://arxiv.org/abs/2602.06843 · HTML: https://mlsanity.com/paper/2602.06843/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 31 references that are in the map.
- First posted on arXiv 6 Feb 2026.

## Builds on (most cited first, 8 of 8)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement](https://mlsanity.com/paper/2409.12122/) (2024)
- [The Linear Representation Hypothesis and the Geometry of Large Language Models](https://mlsanity.com/paper/2311.03658/) (2023)
- [Insights on representational similarity in neural networks with canonical correlation](https://mlsanity.com/paper/1806.05759/) (2018)
- [Towards a Definition of Disentangled Representations](https://mlsanity.com/paper/1812.02230/) (2018)
- [Human Behavioral Benchmarking: Numeric Magnitude Comparison Effects in Large Language Models](https://mlsanity.com/paper/2305.10782/) (2023)
- [Development of Cognitive Intelligence in Pre-trained Language Models](https://mlsanity.com/paper/2407.01047/) (2024)

## Built on by (most cited first, 5 of 5)

- [Decomposing how prompting steers behavior](https://mlsanity.com/paper/2606.03093/) (2026)
- [SemRF: A Semantic Reference Frame for Residual-Stream Dynamics in Language Models](https://mlsanity.com/paper/2606.32022/) (2026)
- [Language Models Represent and Transform Concepts with Shared Geometry](https://mlsanity.com/paper/2607.04525/) (2026)
- [The Geometric Price of Discrete Logic: Context-driven Manifold Dynamics of Number Representations](https://mlsanity.com/paper/2603.23577/) (2026)
- [On the use of foundation models in cognitive science](https://mlsanity.com/paper/2608.07812/) (2026)

## Abstract (arXiv)

A central question in cognitive science is whether conceptual representations converge onto a shared manifold to support generalization, or diverge into orthogonal subspaces to minimize task interference. While prior work has discovered evidence for both, a mechanistic account of how these properties coexist and transform across tasks remains elusive. We propose that representational sharing lies not in the concepts themselves, but in the geometric relations between them. Using number concepts as a testbed and language models as high-dimensional computational substrates, we show that number representations preserve a stable relational structure across tasks. Task-specific representations are embedded in distinct subspaces, with low-level features like magnitude and parity encoded along separable linear directions. Crucially, we find that these subspaces are largely transformable into one another via linear mappings, indicating that representations share relational structure despite being located in distinct subspaces. Together, these results provide a mechanistic lens of how language models balance the shared structure of number representation with functional flexibility. It suggests that understanding arises when task-specific transformations are applied to a shared underlying relational structure of conceptual representations.