# Extraction of Medication Names from Twitter Using Augmentation and an Ensemble of Language Models

arXiv: https://arxiv.org/abs/2111.06664 · HTML: https://mlsanity.com/paper/2111.06664/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 14 references that are in the map.
- First posted on arXiv 12 Nov 2021.

## Builds on (most cited first, 8 of 8)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification](https://mlsanity.com/paper/2010.12421/) (2020)
- [BERTweet: A pre-trained language model for English Tweets](https://mlsanity.com/paper/2005.10200/) (2020)
- [Cost-effective Selection of Pretraining Data: A Case Study of Pretraining BERT on Social Media](https://mlsanity.com/paper/2010.01150/) (2020)
- [TweetBERT: A Pretrained Language Representation Model for Twitter Text Analysis](https://mlsanity.com/paper/2010.11091/) (2020)
- [Deep Neural Networks Ensemble for Detecting Medication Mentions in Tweets](https://mlsanity.com/paper/1904.05308/) (2019)

## Abstract (arXiv)

The BioCreative VII Track 3 challenge focused on the identification of medication names in Twitter user timelines. For our submission to this challenge, we expanded the available training data by using several data augmentation techniques. The augmented data was then used to fine-tune an ensemble of language models that had been pre-trained on general-domain Twitter content. The proposed approach outperformed the prior state-of-the-art algorithm Kusuri and ranked high in the competition for our selected objective function, overlapping F1 score.