# Contextualized Medication Information Extraction Using Transformer-based Deep Learning Architectures

arXiv: https://arxiv.org/abs/2303.08259 · HTML: https://mlsanity.com/paper/2303.08259/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 29 references that are in the map.
- First posted on arXiv 14 Mar 2023.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Publicly Available Clinical BERT Embeddings](https://mlsanity.com/paper/1904.03323/) (2019)
- [GatorTron: A Large Clinical Language Model to Unlock Patient Information from Unstructured Electronic Health Records](https://mlsanity.com/paper/2203.03540/) (2022)
- [Bidirectional Recurrent Neural Networks for Medical Event Detection in Electronic Health Records](https://mlsanity.com/paper/1606.07953/) (2016)
- [Toward Understanding Clinical Context of Medication Change Events in Clinical Narratives](https://mlsanity.com/paper/2011.08835/) (2020)

## Built on by (most cited first, 8 of 8)

- [Decoding ChatGPT: A Taxonomy of Existing Research, Current Challenges, and Possible Future Directions](https://mlsanity.com/paper/2307.14107/) (2023)
- [Large Language Models in Biomedical and Health Informatics: A Review with Bibliometric Analysis](https://mlsanity.com/paper/2403.16303/) (2024)
- [Generative Large Language Models Are All-purpose Text Analytics Engines: Text-to-text Learning Is All Your Need](https://mlsanity.com/paper/2312.06099/) (2023)
- [GAMedX: Generative AI-based Medical Entity Data Extractor Using Large Language Models](https://mlsanity.com/paper/2405.20585/) (2024)
- [Large Language Models for Granularized Barrett's Esophagus Diagnosis Classification](https://mlsanity.com/paper/2308.08660/) (2023)
- [Scalable Medication Extraction and Discontinuation Identification from Electronic Health Records Using Large Language Models](https://mlsanity.com/paper/2506.11137/) (2025)
- [A Study of Large Language Models for Patient Information Extraction: Model Architecture, Fine-Tuning Strategy, and Multi-task Instruction Tuning](https://mlsanity.com/paper/2509.04753/) (2025)
- ["Where does it hurt?" -- Dataset and Study on Physician Intent Trajectories in Doctor Patient Dialogues](https://mlsanity.com/paper/2508.19077/) (2025)

## Abstract (arXiv)

Objective: To develop a natural language processing (NLP) system to extract medications and contextual information that help understand drug changes. This project is part of the 2022 n2c2 challenge. Materials and methods: We developed NLP systems for medication mention extraction, event classification (indicating medication changes discussed or not), and context classification to classify medication changes context into 5 orthogonal dimensions related to drug changes. We explored 6 state-of-the-art pretrained transformer models for the three subtasks, including GatorTron, a large language model pretrained using >90 billion words of text (including >80 billion words from >290 million clinical notes identified at the University of Florida Health). We evaluated our NLP systems using annotated data and evaluation scripts provided by the 2022 n2c2 organizers. Results:Our GatorTron models achieved the best F1-scores of 0.9828 for medication extraction (ranked 3rd), 0.9379 for event classification (ranked 2nd), and the best micro-average accuracy of 0.9126 for context classification. GatorTron outperformed existing transformer models pretrained using smaller general English text and clinical text corpora, indicating the advantage of large language models. Conclusion: This study demonstrated the advantage of using large transformer models for contextual medication information extraction from clinical narratives.