# Numeracy for Language Models: Evaluating and Improving their Ability to Predict Numbers

arXiv: https://arxiv.org/abs/1805.08154 · HTML: https://mlsanity.com/paper/1805.08154/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 May 2018.

## Built on by (most cited first, 30 of 42)

- [Language Models are Multilingual Chain-of-Thought Reasoners](https://mlsanity.com/paper/2210.03057/) (2022)
- [Impact of Pretraining Term Frequencies on Few-Shot Reasoning](https://mlsanity.com/paper/2202.07206/) (2022)
- [Why Tabular Foundation Models Should Be a Research Priority](https://mlsanity.com/paper/2405.01147/) (2024)
- [Representing Numbers in NLP: a Survey and a Vision](https://mlsanity.com/paper/2103.13136/) (2021)
- [Probing for the Usage of Grammatical Number](https://mlsanity.com/paper/2204.08831/) (2022)
- [Leveraging Language Foundation Models for Human Mobility Forecasting](https://mlsanity.com/paper/2209.05479/) (2022)
- [Number Cookbook: Number Understanding of Language Models and How to Improve It](https://mlsanity.com/paper/2411.03766/) (2024)
- [Zero-shot Generalization in Dialog State Tracking through Generative Question Answering](https://mlsanity.com/paper/2101.08333/) (2021)
- [Can Language Beat Numerical Regression? Language-Based Multimodal Trajectory Prediction](https://mlsanity.com/paper/2403.18447/) (2024)
- [Mathematical Language Models: A Survey](https://mlsanity.com/paper/2312.07622/) (2023)
- [How Much Coffee Was Consumed During EMNLP 2019? Fermi Problems: A New Reasoning Challenge for AI](https://mlsanity.com/paper/2110.14207/) (2021)
- [AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling](https://mlsanity.com/paper/2411.17284/) (2024)
- [A Survey on Mathematical Reasoning and Optimization with Large Language Models](https://mlsanity.com/paper/2503.17726/) (2025)
- [Human Behavioral Benchmarking: Numeric Magnitude Comparison Effects in Large Language Models](https://mlsanity.com/paper/2305.10782/) (2023)
- [The Geometry of Numerical Reasoning: Language Models Compare Numeric Properties in Linear Subspaces](https://mlsanity.com/paper/2410.13194/) (2024)
- [LUNA: Language Understanding with Number Augmentations on Transformers via Number Plugins and Pre-training](https://mlsanity.com/paper/2212.02691/) (2022)
- [Towards Robust Numerical Question Answering: Diagnosing Numerical Capabilities of NLP Systems](https://mlsanity.com/paper/2211.07455/) (2022)
- [Masked Measurement Prediction: Learning to Jointly Predict Quantities and Units from Textual Context](https://mlsanity.com/paper/2112.08616/) (2021)
- [Learning Non-linguistic Skills without Sacrificing Linguistic Proficiency](https://mlsanity.com/paper/2305.08246/) (2023)
- [Knowledge of Pretrained Language Models on Surface Information of Tokens](https://mlsanity.com/paper/2402.09808/) (2024)
- [How to Leverage Digit Embeddings to Represent Numbers?](https://mlsanity.com/paper/2407.00894/) (2024)
- [Overcoming Barriers to Skill Injection in Language Modeling: Case Study in Arithmetic](https://mlsanity.com/paper/2211.02098/) (2022)
- [MachineLearningLM: Scaling Many-shot In-context Learning via Continued Pretraining](https://mlsanity.com/paper/2509.06806/) (2025)
- [Agentic Physical AI toward a Domain-Specific Foundation Model for Energy Systems: A Case Study on Nuclear Reactor Control](https://mlsanity.com/paper/2512.23292/) (2025)
- [Do Language Models Understand Measurements?](https://mlsanity.com/paper/2210.12694/) (2022)
- [KitchenScale: Learning to predict ingredient quantities from recipe contexts](https://mlsanity.com/paper/2304.10739/) (2023)
- [GPT Czech Poet: Generation of Czech Poetic Strophes with Language Models](https://mlsanity.com/paper/2407.12790/) (2024)
- [Confidence-Aware Document OCR Error Detection](https://mlsanity.com/paper/2409.04117/) (2024)
- [Do You Know What You Are Talking About? Characterizing Query-Knowledge Relevance For Reliable Retrieval Augmented Generation](https://mlsanity.com/paper/2410.08320/) (2024)
- [Improving Downstream Task Performance by Treating Numbers as Entities](https://mlsanity.com/paper/2205.03559/) (2022)

## Abstract (arXiv)

Numeracy is the ability to understand and work with numbers. It is a necessary skill for composing and understanding documents in clinical, scientific, and other technical domains. In this paper, we explore different strategies for modelling numerals with language models, such as memorisation and digit-by-digit composition, and propose a novel neural architecture that uses a continuous probability density function to model numerals from an open vocabulary. Our evaluation on clinical and scientific datasets shows that using hierarchical models to distinguish numerals from words improves a perplexity metric on the subset of numerals by 2 and 4 orders of magnitude, respectively, over non-hierarchical models. A combination of strategies can further improve perplexity. Our continuous probability density function model reduces mean absolute percentage errors by 18% and 54% in comparison to the second best strategy for each dataset, respectively.