# Generating Medical Prescriptions with Conditional Transformer

arXiv: https://arxiv.org/abs/2310.19727 · HTML: https://mlsanity.com/paper/2310.19727/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 39 references that are in the map.
- First posted on arXiv 30 Oct 2023.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation](https://mlsanity.com/paper/1611.04558/) (2016)
- [Document Ranking with a Pretrained Sequence-to-Sequence Model](https://mlsanity.com/paper/2003.06713/) (2020)
- [GatorTron: A Large Clinical Language Model to Unlock Patient Information from Unstructured Electronic Health Records](https://mlsanity.com/paper/2203.03540/) (2022)
- [Natural Language Generation for Electronic Health Records](https://mlsanity.com/paper/1806.01353/) (2018)
- [Patchwork Learning: A Paradigm Towards Integrative Analysis across Diverse Biomedical Data Sources](https://mlsanity.com/paper/2305.06217/) (2023)
- [Biomedical Data-to-Text Generation via Fine-Tuning Transformers](https://mlsanity.com/paper/2109.01518/) (2021)
- [Controlled Text Generation using T5 based Encoder-Decoder Soft Prompt Tuning and Analysis of the Utility of Generated Text in AI](https://mlsanity.com/paper/2212.02924/) (2022)
- [Mitigating Health Data Poverty: Generative Approaches versus Resampling for Time-series Clinical Data](https://mlsanity.com/paper/2210.13958/) (2022)
- [MedMine: Examining Pre-trained Language Models on Medication Mining](https://mlsanity.com/paper/2308.03629/) (2023)

## Built on by (most cited first, 3 of 3)

- [Light-weight Fine-tuning Method for Defending Adversarial Noise in Pre-trained Medical Vision-Language Models](https://mlsanity.com/paper/2407.02716/) (2024)
- [Generating Synthetic Free-text Medical Records with Low Re-identification Risk using Masked Language Modeling](https://mlsanity.com/paper/2409.09831/) (2024)
- [HealthcareNLP: where are we and what is next?](https://mlsanity.com/paper/2512.08617/) (2025)

## Abstract (arXiv)

Access to real-world medication prescriptions is essential for medical research and healthcare quality improvement. However, access to real medication prescriptions is often limited due to the sensitive nature of the information expressed. Additionally, manually labelling these instructions for training and fine-tuning Natural Language Processing (NLP) models can be tedious and expensive. We introduce a novel task-specific model architecture, Label-To-Text-Transformer (LT3), tailored to generate synthetic medication prescriptions based on provided labels, such as a vocabulary list of medications and their attributes. LT3 is trained on a set of around 2K lines of medication prescriptions extracted from the MIMIC-III database, allowing the model to produce valuable synthetic medication prescriptions. We evaluate LT3's performance by contrasting it with a state-of-the-art Pre-trained Language Model (PLM), T5, analysing the quality and diversity of generated texts. We deploy the generated synthetic data to train the SpacyNER model for the Named Entity Recognition (NER) task over the n2c2-2018 dataset. The experiments show that the model trained on synthetic data can achieve a 96-98% F1 score at Label Recognition on Drug, Frequency, Route, Strength, and Form. LT3 codes and data will be shared at urlhttps://github.com/HECTA-UoM/Label-To-Text-Transformer