# Improving Downstream Task Performance by Treating Numbers as Entities

arXiv: https://arxiv.org/abs/2205.03559 · HTML: https://mlsanity.com/paper/2205.03559/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 25 references that are in the map.
- First posted on arXiv 7 May 2022.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016)
- [Do NLP Models Know Numbers? Probing Numeracy in Embeddings](https://mlsanity.com/paper/1909.07940/) (2019)
- [Injecting Numerical Reasoning Skills into Language Models](https://mlsanity.com/paper/2004.04487/) (2020)
- [Joint Embedding of Words and Labels for Text Classification](https://mlsanity.com/paper/1805.04174/) (2018)
- [Do Language Embeddings Capture Scales?](https://mlsanity.com/paper/2010.05345/) (2020)
- [Giving BERT a Calculator: Finding Operations and Arguments with Reading Comprehension](https://mlsanity.com/paper/1909.00109/) (2019)
- [Numeracy for Language Models: Evaluating and Improving their Ability to Predict Numbers](https://mlsanity.com/paper/1805.08154/) (2018)
- [An Empirical Investigation of Contextualized Number Prediction](https://mlsanity.com/paper/2011.07961/) (2020)
- [Syntax-Infused Transformer and BERT models for Machine Translation and Natural Language Understanding](https://mlsanity.com/paper/1911.06156/) (2019)
- [Learning Compressed Sentence Representations for On-Device Text Processing](https://mlsanity.com/paper/1906.08340/) (2019)
- [How do lexical semantics affect translation? An empirical study](https://mlsanity.com/paper/2201.00075/) (2021)
- [Unsupervised Numerical Reasoning to Extract Phenotypes from Clinical Text by Leveraging External Knowledge](https://mlsanity.com/paper/2204.10202/) (2022)

## Built on by (most cited first, 1 of 1)

- [Exploring Internal Numeracy in Language Models: A Case Study on ALBERT](https://mlsanity.com/paper/2404.16574/) (2024)

## Abstract (arXiv)

Numbers are essential components of text, like any other word tokens, from which natural language processing (NLP) models are built and deployed. Though numbers are typically not accounted for distinctly in most NLP tasks, there is still an underlying amount of numeracy already exhibited by NLP models. In this work, we attempt to tap this potential of state-of-the-art NLP models and transfer their ability to boost performance in related tasks. Our proposed classification of numbers into entities helps NLP models perform well on several tasks, including a handcrafted Fill-In-The-Blank (FITB) task and on question answering using joint embeddings, outperforming the BERT and RoBERTa baseline classification.