# Masked Measurement Prediction: Learning to Jointly Predict Quantities and Units from Textual Context

arXiv: https://arxiv.org/abs/2112.08616 · HTML: https://mlsanity.com/paper/2112.08616/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 32 references that are in the map.
- First posted on arXiv 16 Dec 2021.

## Builds on (most cited first, 20 of 20)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs](https://mlsanity.com/paper/1903.00161/) (2019)
- [FinQA: A Dataset of Numerical Reasoning over Financial Data](https://mlsanity.com/paper/2109.00122/) (2021)
- [TabFact: A Large-scale Dataset for Table-based Fact Verification](https://mlsanity.com/paper/1909.02164/) (2019)
- [Do NLP Models Know Numbers? Probing Numeracy in Embeddings](https://mlsanity.com/paper/1909.07940/) (2019)
- [Injecting Numerical Reasoning Skills into Language Models](https://mlsanity.com/paper/2004.04487/) (2020)
- [Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-trained Language Models](https://mlsanity.com/paper/2005.00683/) (2020)
- [Representing Numbers in NLP: a Survey and a Vision](https://mlsanity.com/paper/2103.13136/) (2021)
- [Do Language Embeddings Capture Scales?](https://mlsanity.com/paper/2010.05345/) (2020)
- [NumNet: Machine Reading Comprehension with Numerical Reasoning](https://mlsanity.com/paper/1910.06701/) (2019)
- [Numeracy for Language Models: Evaluating and Improving their Ability to Predict Numbers](https://mlsanity.com/paper/1805.08154/) (2018)
- [EQUATE: A Benchmark Evaluation Framework for Quantitative Reasoning in Natural Language Inference](https://mlsanity.com/paper/1901.03735/) (2019)
- [How Large Are Lions? Inducing Distributions over Quantitative Attributes](https://mlsanity.com/paper/1906.01327/) (2019)
- [How Much Coffee Was Consumed During EMNLP 2019? Fermi Problems: A New Reasoning Challenge for AI](https://mlsanity.com/paper/2110.14207/) (2021)
- [An Empirical Investigation of Contextualized Number Prediction](https://mlsanity.com/paper/2011.07961/) (2020)
- [NumGPT: Improving Numeracy Ability of Generative Pre-trained Models](https://mlsanity.com/paper/2109.03137/) (2021)
- [Learning to Reason for Text Generation from Scientific Tables](https://mlsanity.com/paper/2104.08296/) (2021)
- [Investigating Numeracy Learning Ability of a Text-to-Text Transfer Model](https://mlsanity.com/paper/2109.04672/) (2021)
- [Where's My Head? Definition, Dataset and Models for Numeric Fused-Heads Identification and Resolution](https://mlsanity.com/paper/1905.10886/) (2019)
- [How Much is 131 Million Dollars? Putting Numbers in Perspective with Compositional Descriptions](https://mlsanity.com/paper/1609.00070/) (2016)

## Built on by (most cited first, 5 of 5)

- [A Mechanistic Interpretation of Arithmetic Reasoning in Language Models using Causal Mediation Analysis](https://mlsanity.com/paper/2305.15054/) (2023)
- [A Causal Framework to Quantify the Robustness of Mathematical Reasoning with Language Models](https://mlsanity.com/paper/2210.12023/) (2022)
- [KitchenScale: Learning to predict ingredient quantities from recipe contexts](https://mlsanity.com/paper/2304.10739/) (2023)
- [Numbers Matter! Bringing Quantity-awareness to Retrieval Systems](https://mlsanity.com/paper/2407.10283/) (2024)
- [Language Models Compare Quantities Using Number-specific and Unit-specific Heuristics](https://mlsanity.com/paper/2606.03982/) (2026)

## Abstract (arXiv)

Physical measurements constitute a large portion of numbers in academic papers, engineering reports, and web tables. Current benchmarks fall short of properly evaluating numeracy of pretrained language models on measurements, hindering research on developing new methods and applying them to numerical tasks. To that end, we introduce a novel task, Masked Measurement Prediction (MMP), where a model learns to reconstruct a number together with its associated unit given masked text. MMP is useful for both training new numerically informed models as well as evaluating numeracy of existing systems. In order to address this task, we introduce a new Generative Masked Measurement (GeMM) model that jointly learns to predict numbers along with their units. We perform fine-grained analyses comparing our model with various ablations and baselines. We use linear probing of traditional pretrained transformer models (RoBERTa) to show that they significantly underperform jointly trained number-unit models, highlighting the difficulty of this new task and the benefits of our proposed pretraining approach. We hope this framework accelerates the progress towards building more robust numerical reasoning systems in the future.