# EHRmonize: A Framework for Medical Concept Abstraction from Electronic Health Records using Large Language Models

arXiv: https://arxiv.org/abs/2407.00242 · HTML: https://mlsanity.com/paper/2407.00242/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 23 references that are in the map.
- First posted on arXiv 28 Jun 2024.

## Builds on (most cited first, 6 of 6)

- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models](https://mlsanity.com/paper/2304.01852/) (2023)
- [MIMIC-Extract: A Data Extraction, Preprocessing, and Representation Pipeline for MIMIC-III](https://mlsanity.com/paper/1907.08322/) (2019)
- [Generative Text-Guided 3D Vision-Language Pretraining for Unified Medical Image Segmentation](https://mlsanity.com/paper/2306.04811/) (2023)
- [Language Models are Surprisingly Fragile to Drug Names in Biomedical Benchmarks](https://mlsanity.com/paper/2406.12066/) (2024)
- [Cross-Care: Assessing the Healthcare Implications of Pre-training Data on Language Model Bias](https://mlsanity.com/paper/2405.05506/) (2024)

## Built on by (most cited first, 4 of 4)

- [A Collection of Innovations in Medical AI for patient records in 2024](https://mlsanity.com/paper/2503.05768/) (2025)
- [Ontology- and LLM-based Data Harmonization for Federated Learning in Healthcare](https://mlsanity.com/paper/2505.20020/) (2025)
- [Predicting Failures of LLMs to Link Biomedical Ontology Terms to Identifiers Evidence Across Models and Ontologies](https://mlsanity.com/paper/2509.04458/) (2025)
- [Evaluating Large Language Models for Automated Clinical Abstraction in Pulmonary Embolism Registries: Performance Across Model Sizes, Versions, and Parameters](https://mlsanity.com/paper/2503.21004/) (2025)

## Abstract (arXiv)

Electronic health records (EHRs) contain vast amounts of complex data, but harmonizing and processing this information remains a challenging and costly task requiring significant clinical expertise. While large language models (LLMs) have shown promise in various healthcare applications, their potential for abstracting medical concepts from EHRs remains largely unexplored. We introduce EHRmonize, a framework leveraging LLMs to abstract medical concepts from EHR data. Our study uses medication data from two real-world EHR databases to evaluate five LLMs on two free-text extraction and six binary classification tasks across various prompting strategies. GPT-4o's with 10-shot prompting achieved the highest performance in all tasks, accompanied by Claude-3.5-Sonnet in a subset of tasks. GPT-4o achieved an accuracy of 97% in identifying generic route names, 82% for generic drug names, and 100% in performing binary classification of antibiotics. While EHRmonize significantly enhances efficiency, reducing annotation time by an estimated 60%, we emphasize that clinician oversight remains essential. Our framework, available as a Python package, offers a promising tool to assist clinicians in EHR data abstraction, potentially accelerating healthcare research and improving data harmonization processes.