# Weber's Law in Transformer Magnitude Representations: Efficient Coding, Representational Geometry, and Psychophysical Laws in Language Models

arXiv: https://arxiv.org/abs/2603.20642 · HTML: https://mlsanity.com/paper/2603.20642/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 19 references that are in the map.
- First posted on arXiv 21 Mar 2026.

## Builds on (most cited first, 6 of 6)

- [Language Models Represent Space and Time](https://mlsanity.com/paper/2310.02207/) (2023)
- [Representing Numbers in NLP: a Survey and a Vision](https://mlsanity.com/paper/2103.13136/) (2021)
- [Large language models predict human sensory judgments across six modalities](https://mlsanity.com/paper/2302.01308/) (2023)
- [Language Models Encode Numbers Using Digit Representations in Base 10](https://mlsanity.com/paper/2410.11781/) (2024)
- [What is a Number, That a Large Language Model May Know It?](https://mlsanity.com/paper/2502.01540/) (2025)
- [Language Models Learn Universal Representations of Numbers and Here's Why You Should Care](https://mlsanity.com/paper/2510.26285/) (2025)

## Built on by (most cited first, 4 of 4)

- [The Metacognitive Monitoring Battery: A Cross-Domain Benchmark for LLM Self-Monitoring](https://mlsanity.com/paper/2604.15702/) (2026)
- [Same Geometry, Opposite Noise: Transformer Magnitude Representations Lack Scalar Variability](https://mlsanity.com/paper/2604.04469/) (2026)
- [Temporal Preference Concepts and their Functions in a Large Language Model](https://mlsanity.com/paper/2606.05194/) (2026)
- [Poor Man's Agentic Modeling: Simulating Large LLM-Agent Societies on a Laptop](https://mlsanity.com/paper/2608.11215/) (2026)

## Abstract (arXiv)

How do transformer language models represent magnitude? Recent work disagrees: some find logarithmic spacing, others linear encoding, others per-digit circular representations. We apply the formal tools of psychophysics to resolve this. Using four converging paradigms (representational similarity analysis, behavioural discrimination, precision gradients, causal intervention) across three magnitude domains in three 7-9B instruction-tuned models spanning three architecture families (Llama, Mistral, Qwen), we report three findings. First, representational geometry is consistently log-compressive: RSA correlations with a Weber-law dissimilarity matrix ranged from .68 to .96 across all 96 model-domain-layer cells, with linear geometry never preferred. Second, this geometry is dissociated from behaviour: one model produces a human-range Weber fraction (WF = 0.20) while the other does not, and both models perform at chance on temporal and spatial discrimination despite possessing logarithmic geometry. Third, causal intervention reveals a layer dissociation: early layers are functionally implicated in magnitude processing (4.1x specificity) while later layers where geometry is strongest are not causally engaged (1.2x). Corpus analysis confirms the efficient coding precondition (alpha = 0.77). These results suggest that training data statistics alone are sufficient to produce log-compressive magnitude geometry, but geometry alone does not guarantee behavioural competence.