Improving Downstream Task Performance by Treating Numbers as Entities
Not on the map. Why?
Abstract (arXiv)
Numbers are essential components of text, like any other word tokens, from which natural language processing (NLP) models are built and deployed. Though numbers are typically not accounted for distinctly in most NLP tasks, there is still an underlying amount of numeracy already exhibited by NLP models. In this work, we attempt to tap this potential of state-of-the-art NLP models and transfer their ability to boost performance in related tasks. Our proposed classification of numbers into entities helps NLP models perform well on several tasks, including a handcrafted Fill-In-The-Blank (FITB) task and on question answering using joint embeddings, outperforming the BERT and RoBERTa baseline classification.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 17 of its 25 references that are in the map. First posted on arXiv 7 May 2022.
Builds on
17 of 25 references.
References · 17
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
- 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
- 2016Neural Architectures for Named Entity Recognitioncited by 550
- 2019Do NLP Models Know Numbers? Probing Numeracy in Embeddingscited by 159
- 2020Injecting Numerical Reasoning Skills into Language Modelscited by 133
- 2018Joint Embedding of Words and Labels for Text Classificationcited by 69
- 2020Do Language Embeddings Capture Scales?cited by 63
- 2019Giving BERT a Calculator: Finding Operations and Arguments with Reading Comprehensioncited by 51
- 2018Numeracy for Language Models: Evaluating and Improving their Ability to Predict Numberscited by 42
5 more
- 2020An Empirical Investigation of Contextualized Number Predictioncited by 21
- 2019Syntax-Infused Transformer and BERT models for Machine Translation and Natural Language Understandingcited by 20
- 2019Learning Compressed Sentence Representations for On-Device Text Processingcited by 5
- 2021How do lexical semantics affect translation? An empirical studycited by 2
- 2022Unsupervised Numerical Reasoning to Extract Phenotypes from Clinical Text by Leveraging External Knowledgecited by 2
Built on by
Citing papers per year: 2024: 1.
1 paper.