Do Language Models Understand Measurements?
Not on the map. Why?
Abstract (arXiv)
Recent success of pre-trained language models (PLMs) has stimulated interest in their ability to understand and work with numbers. Yet, the numerical reasoning over measurements has not been formally studied despite their importance. In this study, we show that PLMs lack the capability required for reasoning over measurements. Furthermore, we find that a language model trained on a measurement-rich corpus shows better performance on understanding measurements. We propose a simple embedding strategy to better distinguish between numbers and units, which leads to a significant improvement in the probing tasks.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 20 of its 26 references that are in the map. First posted on arXiv 23 Oct 2022.
Builds on
20 of 26 references.
References · 20
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
- 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
- 2019Language Models as Knowledge Bases?cited by 1,922
- 2019How Can We Know What Language Models Know?cited by 953
- 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
- 2019oLMpics -- On what Language Model Pre-training Capturescited by 161
- 2019Do NLP Models Know Numbers? Probing Numeracy in Embeddingscited by 157
- 2020Injecting Numerical Reasoning Skills into Language Modelscited by 133
- 2020Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-trained Language Modelscited by 110
- 2020Do Language Embeddings Capture Scales?cited by 63
8 more
- 2018Numeracy for Language Models: Evaluating and Improving their Ability to Predict Numberscited by 42
- 2020An Empirical Study of Multi-Task Learning on BERT for Biomedical Text Miningcited by 35
- 2017Verb Physics: Relative Physical Knowledge of Actions and Objectscited by 26
- 2019How Large Are Lions? Inducing Distributions over Quantitative Attributescited by 26
- 2021NumGPT: Improving Numeracy Ability of Generative Pre-trained Modelscited by 15
- 2021Investigating Numeracy Learning Ability of a Text-to-Text Transfer Modelcited by 13
- 2021NT5?! Training T5 to Perform Numerical Reasoningcited by 11
- 2020Probing for Multilingual Numerical Understanding in Transformer-Based Language Modelscited by 10
Built on by
Citing papers per year: 2024: 2.
2 papers.