mlsanityData to 14 Oct 2026

Language Models Compare Quantities Using Number-specific and Unit-specific Heuristics

arXiv 2606.03982 ↗First posted 2 Jun 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Quantities with measurement units, such as 110 cm and 1.2 m, require language models (LMs) to combine a numeral with a symbolic unit scale. Here, we study how LMs compare such quantities in controlled settings spanning several unit systems. We find that accuracy degrades near the comparison boundary, where small changes in value determine the correct answer. The resulting errors are systematic: linear surrogate models predict LM preferences from numerical-difference and unit-scale-difference cues, and causal interventions on subspaces aligned with these variables shift model's output. The results suggest that LMs compare quantities through a bag of heuristics over numerals and units, rather than first converting both expressions to an exact shared-scale representation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 25 references that are in the map. First posted on arXiv 2 Jun 2026.

Builds on

16 of 25 references.

References · 16

  1. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
  2. 2023Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited by 209
  3. 2023The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networkscited by 156
  4. 2024Arithmetic Without Algorithms: Language Models Solve Math With a Bag of Heuristicscited by 97
  5. 2025Language Models Use Trigonometry to Do Additioncited by 70
  6. 2024Interpreting and Improving Large Language Models in Arithmetic Calculationcited by 45
  7. 2024Pre-trained Large Language Models Use Fourier Features to Compute Additioncited by 39
  8. 2024Language Models Encode Numbers Using Digit Representations in Base 10cited by 32
  9. 2024Language Models Encode the Value of Numbers Linearlycited by 20
  10. 2024The Geometry of Numerical Reasoning: Language Models Compare Numeric Properties in Linear Subspacescited by 10
  11. 2024Understanding Addition and Subtraction in Transformerscited by 8
  12. 2024Survival of the Fittest Representation: A Case Study with Modular Additioncited by 7
4 more
  1. 2026LLMs Know More About Numbers than They Can Saycited by 7
  2. 2021Masked Measurement Prediction: Learning to Jointly Predict Quantities and Units from Textual Contextcited by 5
  3. 2025On Generalization across Measurement Systems: LLMs Entail More Test-Time Compute for Underrepresented Culturescited by 5
  4. 2023Enhancing Quantitative Reasoning Skills of Large Language Models through Dimension Perceptioncited by 4

Built on by

Not cited yet.