# Neural Medication Extraction: A Comparison of Recent Models in Supervised and Semi-supervised Learning Settings

arXiv: https://arxiv.org/abs/2110.10213 · HTML: https://mlsanity.com/paper/2110.10213/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 32 references that are in the map.
- First posted on arXiv 19 Oct 2021.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [Publicly Available Clinical BERT Embeddings](https://mlsanity.com/paper/1904.03323/) (2019)
- [Convolutional Sequence to Sequence Learning](https://mlsanity.com/paper/1705.03122/) (2017)
- [Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets](https://mlsanity.com/paper/1906.05474/) (2019)
- [Enhancing Clinical Concept Extraction with Contextual Embeddings](https://mlsanity.com/paper/1902.08691/) (2019)
- [MT-Clinical BERT: Scaling Clinical Information Extraction with Multitask Learning](https://mlsanity.com/paper/2004.10220/) (2020)
- [Semi-Supervised Neural Text Generation by Joint Learning of Natural Language Generation and Natural Language Understanding Models](https://mlsanity.com/paper/1910.03484/) (2019)
- [A Data-driven Approach for Noise Reduction in Distantly Supervised Biomedical Relation Extraction](https://mlsanity.com/paper/2005.12565/) (2020)
- [Exploring Semi-supervised Variational Autoencoders for Biomedical Relation Extraction](https://mlsanity.com/paper/1901.06103/) (2019)
- [Assessment of Amazon Comprehend Medical: Medication Information Extraction](https://mlsanity.com/paper/2002.00481/) (2020)

## Built on by (most cited first, 2 of 2)

- [Extracting Medication Changes in Clinical Narratives using Pre-trained Language Models](https://mlsanity.com/paper/2208.08417/) (2022)
- [A Spoken Drug Prescription Dataset in French for Spoken Language Understanding](https://mlsanity.com/paper/2207.08292/) (2022)

## Abstract (arXiv)

Drug prescriptions are essential information that must be encoded in electronic medical records. However, much of this information is hidden within free-text reports. This is why the medication extraction task has emerged. To date, most of the research effort has focused on small amount of data and has only recently considered deep learning methods. In this paper, we present an independent and comprehensive evaluation of state-of-the-art neural architectures on the I2B2 medical prescription extraction task both in the supervised and semi-supervised settings. The study shows the very competitive performance of simple DNN models on the task as well as the high interest of pre-trained models. Adapting the latter models on the I2B2 dataset enables to push medication extraction performances above the state-of-the-art. Finally, the study also confirms that semi-supervised techniques are promising to leverage large amounts of unlabeled data in particular in low resource setting when labeled data is too costly to acquire.