# Number Representations in LLMs: A Computational Parallel to Human Perception

arXiv: https://arxiv.org/abs/2502.16147 · HTML: https://mlsanity.com/paper/2502.16147/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 22 references that are in the map.
- First posted on arXiv 22 Feb 2025.

## Builds on (most cited first, 11 of 11)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Do Language Embeddings Capture Scales?](https://mlsanity.com/paper/2010.05345/) (2020)
- [Identifying and Mitigating Vulnerabilities in LLM-Integrated Applications](https://mlsanity.com/paper/2311.16153/) (2023)
- [Language Models Encode Numbers Using Digit Representations in Base 10](https://mlsanity.com/paper/2410.11781/) (2024)
- [Language Models Encode the Value of Numbers Linearly](https://mlsanity.com/paper/2401.03735/) (2024)
- [On the Scaling Laws of Geographical Representation in Language Models](https://mlsanity.com/paper/2402.19406/) (2024)
- [The Geometry of Numerical Reasoning: Language Models Compare Numeric Properties in Linear Subspaces](https://mlsanity.com/paper/2410.13194/) (2024)
- [NICE: To Optimize In-Context Examples or Not?](https://mlsanity.com/paper/2402.06733/) (2024)
- [Scaling Behavior for Large Language Models regarding Numeral Systems: An Example using Pythia](https://mlsanity.com/paper/2409.17391/) (2024)

## Built on by (most cited first, 4 of 4)

- [When Models Manipulate Manifolds: The Geometry of a Counting Task](https://mlsanity.com/paper/2601.04480/) (2026)
- [When Is a Steerable Concept Representation Real? Measurement Confounds in a Cross-Family Audit of Neuroscience Parallels in LLMs](https://mlsanity.com/paper/2608.08159/) (2026)
- [DriveCode: Domain Specific Numerical Encoding for LLM-Based Autonomous Driving](https://mlsanity.com/paper/2603.00919/) (2026)
- [NumerosityVLM: A Cognitively Inspired Benchmark for Interpreting Numerosity Representations in Vision-Language Models](https://mlsanity.com/paper/2608.15425/) (2026)

## Abstract (arXiv)

Humans are believed to perceive numbers on a logarithmic mental number line, where smaller values are represented with greater resolution than larger ones. This cognitive bias, supported by neuroscience and behavioral studies, suggests that numerical magnitudes are processed in a sublinear fashion rather than on a uniform linear scale. Inspired by this hypothesis, we investigate whether large language models (LLMs) exhibit a similar logarithmic-like structure in their internal numerical representations. By analyzing how numerical values are encoded across different layers of LLMs, we apply dimensionality reduction techniques such as PCA and PLS followed by geometric regression to uncover latent structures in the learned embeddings. Our findings reveal that the model's numerical representations exhibit sublinear spacing, with distances between values aligning with a logarithmic scale. This suggests that LLMs, much like humans, may encode numbers in a compressed, non-uniform manner.