# MedMine: Examining Pre-trained Language Models on Medication Mining

arXiv: https://arxiv.org/abs/2308.03629 · HTML: https://mlsanity.com/paper/2308.03629/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 40 references that are in the map.
- First posted on arXiv 7 Aug 2023.

## Builds on (most cited first, 7 of 7)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Multi-domain Clinical Natural Language Processing with MedCAT: the Medical Concept Annotation Toolkit](https://mlsanity.com/paper/2010.01165/) (2020)
- [Med7: a transferable clinical natural language processing model for electronic health records](https://mlsanity.com/paper/2003.01271/) (2020)
- [Examining Large Pre-Trained Language Models for Machine Translation: What You Don't Know About It](https://mlsanity.com/paper/2209.07417/) (2022)
- [Investigating Massive Multilingual Pre-Trained Machine Translation Models for Clinical Domain via Transfer Learning](https://mlsanity.com/paper/2210.06068/) (2022)
- [Exploring the Value of Pre-trained Language Models for Clinical Named Entity Recognition](https://mlsanity.com/paper/2210.12770/) (2022)

## Built on by (most cited first, 2 of 2)

- [Generating Medical Prescriptions with Conditional Transformer](https://mlsanity.com/paper/2310.19727/) (2023)
- [Extraction of Medication and Temporal Relation from Clinical Text using Neural Language Models](https://mlsanity.com/paper/2310.02229/) (2023)

## Abstract (arXiv)

Automatic medication mining from clinical and biomedical text has become a popular topic due to its real impact on healthcare applications and the recent development of powerful language models (LMs). However, fully-automatic extraction models still face obstacles to be overcome such that they can be deployed directly into clinical practice for better impacts. Such obstacles include their imbalanced performances on different entity types and clinical events. In this work, we examine current state-of-the-art pre-trained language models (PLMs) on such tasks, via fine-tuning including the monolingual model Med7 and multilingual large language model (LLM) XLM-RoBERTa. We compare their advantages and drawbacks using historical medication mining shared task data sets from n2c2-2018 challenges. We report the findings we get from these fine-tuning experiments such that they can facilitate future research on addressing them, for instance, how to combine their outputs, merge such models, or improve their overall accuracy by ensemble learning and data augmentation. MedMine is part of the M3 Initiative urlhttps://github.com/HECTA-UoM/M3