# Autocompletion of Chief Complaints in the Electronic Health Records using Large Language Models

arXiv: https://arxiv.org/abs/2401.06088 · HTML: https://mlsanity.com/paper/2401.06088/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 57 references that are in the map.
- First posted on arXiv 11 Jan 2024.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Publicly Available Clinical BERT Embeddings](https://mlsanity.com/paper/1904.03323/) (2019)
- [BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining](https://mlsanity.com/paper/2210.10341/) (2022)
- [Generalizing from a Few Examples: A Survey on Few-Shot Learning](https://mlsanity.com/paper/1904.05046/) (2019)
- [Evaluation of Text Generation: A Survey](https://mlsanity.com/paper/2006.14799/) (2020)
- [On the Automatic Generation of Medical Imaging Reports](https://mlsanity.com/paper/1711.08195/) (2017)
- [Exploring the Limits of Language Modeling](https://mlsanity.com/paper/1602.02410/) (2016)
- [Does Synthetic Data Generation of LLMs Help Clinical Text Mining?](https://mlsanity.com/paper/2303.04360/) (2023)
- [Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems](https://mlsanity.com/paper/1508.01745/) (2015)
- [On the State of the Art of Evaluation in Neural Language Models](https://mlsanity.com/paper/1707.05589/) (2017)
- [Pre-trained Language Models in Biomedical Domain: A Systematic Survey](https://mlsanity.com/paper/2110.05006/) (2021)
- [Generating SOAP Notes from Doctor-Patient Conversations Using Modular Summarization Techniques](https://mlsanity.com/paper/2005.01795/) (2020)
- [Natural Language Generation for Electronic Health Records](https://mlsanity.com/paper/1806.01353/) (2018)
- [A deep language model for software code](https://mlsanity.com/paper/1608.02715/) (2016)
- [Generation of Synthetic Electronic Medical Record Text](https://mlsanity.com/paper/1812.02793/) (2018)
- [AutoMeTS: The Autocomplete for Medical Text Simplification](https://mlsanity.com/paper/2010.10573/) (2020)
- [Measuring Sentences Similarity: A Survey](https://mlsanity.com/paper/1910.03940/) (2019)
- [Learning to Write Notes in Electronic Health Records](https://mlsanity.com/paper/1808.02622/) (2018)
- [Clinical Text Prediction with Numerically Grounded Conditional Language Models](https://mlsanity.com/paper/1610.06370/) (2016)
- [Medical Scientific Table-to-Text Generation with Human-in-the-Loop under the Data Sparsity Constraint](https://mlsanity.com/paper/2205.12368/) (2022)

## Built on by (most cited first, 4 of 4)

- [Prompt engineering paradigms for medical applications: scoping review and recommendations for better practices](https://mlsanity.com/paper/2405.01249/) (2024)
- [How Reliable AI Chatbots are for Disease Prediction from Patient Complaints?](https://mlsanity.com/paper/2405.13219/) (2024)
- [LLM-based Prompt Ensemble for Reliable Medical Entity Recognition from EHRs](https://mlsanity.com/paper/2505.08704/) (2025)
- [Stochastic Parrots or ICU Experts? Large Language Models in Critical Care Medicine: A Scoping Review](https://mlsanity.com/paper/2407.19256/) (2024)

## Abstract (arXiv)

The Chief Complaint (CC) is a crucial component of a patient's medical record as it describes the main reason or concern for seeking medical care. It provides critical information for healthcare providers to make informed decisions about patient care. However, documenting CCs can be time-consuming for healthcare providers, especially in busy emergency departments. To address this issue, an autocompletion tool that suggests accurate and well-formatted phrases or sentences for clinical notes can be a valuable resource for triage nurses. In this study, we utilized text generation techniques to develop machine learning models using CC data. In our proposed work, we train a Long Short-Term Memory (LSTM) model and fine-tune three different variants of Biomedical Generative Pretrained Transformers (BioGPT), namely microsoft/biogpt, microsoft/BioGPT-Large, and microsoft/BioGPT-Large-PubMedQA. Additionally, we tune a prompt by incorporating exemplar CC sentences, utilizing the OpenAI API of GPT-4. We evaluate the models' performance based on the perplexity score, modified BERTScore, and cosine similarity score. The results show that BioGPT-Large exhibits superior performance compared to the other models. It consistently achieves a remarkably low perplexity score of 1.65 when generating CC, whereas the baseline LSTM model achieves the best perplexity score of 170. Further, we evaluate and assess the proposed models' performance and the outcome of GPT-4.0. Our study demonstrates that utilizing LLMs such as BioGPT, leads to the development of an effective autocompletion tool for generating CC documentation in healthcare settings.