# LLM-based Prompt Ensemble for Reliable Medical Entity Recognition from EHRs

arXiv: https://arxiv.org/abs/2505.08704 · HTML: https://mlsanity.com/paper/2505.08704/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 20 references that are in the map.
- First posted on arXiv 13 May 2025.

## Builds on (most cited first, 9 of 9)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Publicly Available Clinical BERT Embeddings](https://mlsanity.com/paper/1904.03323/) (2019)
- [GPT-NER: Named Entity Recognition via Large Language Models](https://mlsanity.com/paper/2304.10428/) (2023)
- [LLM on FHIR -- Demystifying Health Records](https://mlsanity.com/paper/2402.01711/) (2024)
- [How Reliable AI Chatbots are for Disease Prediction from Patient Complaints?](https://mlsanity.com/paper/2405.13219/) (2024)
- [Autocompletion of Chief Complaints in the Electronic Health Records using Large Language Models](https://mlsanity.com/paper/2401.06088/) (2024)

## Built on by (most cited first, 3 of 3)

- [FiNERweb: Datasets and Artifacts for Scalable Multilingual Named Entity Recognition](https://mlsanity.com/paper/2512.13884/) (2025)
- [Improving ARDS Diagnosis Through Context-Aware Concept Bottleneck Models](https://mlsanity.com/paper/2508.09719/) (2025)
- [multiMentalRoBERTa: A Fine-tuned Multiclass Classifier for Mental Health Disorder](https://mlsanity.com/paper/2511.04698/) (2025)

## Abstract (arXiv)

Electronic Health Records (EHRs) are digital records of patient information, often containing unstructured clinical text. Named Entity Recognition (NER) is essential in EHRs for extracting key medical entities like problems, tests, and treatments to support downstream clinical applications. This paper explores prompt-based medical entity recognition using large language models (LLMs), specifically GPT-4o and DeepSeek-R1, guided by various prompt engineering techniques, including zero-shot, few-shot, and an ensemble approach. Among all strategies, GPT-4o with prompt ensemble achieved the highest classification performance with an F1-score of 0.95 and recall of 0.98, outperforming DeepSeek-R1 on the task. The ensemble method improved reliability by aggregating outputs through embedding-based similarity and majority voting.