mlsanityData to 14 Oct 2026

Categorical Perception in Large Language Model Hidden States: Structural Warping at Digit-Count Boundaries

arXiv 2603.28258 ↗First posted 30 Mar 2026cs.CL cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Categorical perception (CP) -- enhanced discriminability at category boundaries -- is among the most studied phenomena in perceptual psychology. This paper reports that analogous geometric warping occurs in the hidden-state representations of large language models (LLMs) processing Arabic numerals. Using representational similarity analysis across six models from five architecture families, the study finds that a CP-additive model (log-distance plus a boundary boost) fits the representational geometry better than a purely continuous model at 100% of primary layers in every model tested. The effect is specific to structurally defined boundaries (digit-count transitions at 10 and 100), absent at non-boundary control positions, and absent in the temperature domain where linguistic categories (hot/cold) lack a tokenisation discontinuity. Two qualitatively distinct signatures emerge: "classic CP" (Gemma, Qwen), where models both categorise explicitly and show geometric warping, and "structural CP" (Llama, Mistral, Phi), where geometry warps at the boundary but models cannot report the category distinction. This dissociation is stable across boundaries and is a property of the architecture, not the stimulus. Structural input-format discontinuities are sufficient to produce categorical perception geometry in LLMs, independently of explicit semantic category knowledge.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 4 of its 23 references that are in the map. First posted on arXiv 30 Mar 2026.

Builds on

4 of 23 references.

References · 4

  1. 2024The Geometry of Categorical and Hierarchical Concepts in Large Language Modelscited by 159
  2. 2024Language Models Encode the Value of Numbers Linearlycited by 20
  3. 2023Human Behavioral Benchmarking: Numeric Magnitude Comparison Effects in Large Language Modelscited by 10
  4. 2020Categorical Perception: A Groundwork for Deep Learningcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.