# Generalizable and Scalable Multistage Biomedical Concept Normalization Leveraging Large Language Models

arXiv: https://arxiv.org/abs/2405.15122 · HTML: https://mlsanity.com/paper/2405.15122/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 58 references that are in the map.
- First posted on arXiv 24 May 2024.

## Builds on (most cited first, 19 of 19)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Large Language Models for Generative Information Extraction: A Survey](https://mlsanity.com/paper/2312.17617/) (2023)
- [Can Foundation Models Wrangle Your Data?](https://mlsanity.com/paper/2205.09911/) (2022)
- [ChatGPT Makes Medicine Easy to Swallow: An Exploratory Case Study on Simplified Radiology Reports](https://mlsanity.com/paper/2212.14882/) (2022)
- [Diagnostic Reasoning Prompts Reveal the Potential for Large Language Model Interpretability in Medicine](https://mlsanity.com/paper/2308.06834/) (2023)
- [MedMentions: A Large Biomedical Corpus Annotated with UMLS Concepts](https://mlsanity.com/paper/1902.09476/) (2019)
- [LLMs4OL: Large Language Models for Ontology Learning](https://mlsanity.com/paper/2307.16648/) (2023)
- [OLaLa: Ontology Matching with Large Language Models](https://mlsanity.com/paper/2311.03837/) (2023)
- [Entity Matching using Large Language Models](https://mlsanity.com/paper/2310.11244/) (2023)
- [Using ChatGPT for Entity Matching](https://mlsanity.com/paper/2305.03423/) (2023)
- [SECNLP: A Survey of Embeddings in Clinical Natural Language Processing](https://mlsanity.com/paper/1903.01039/) (2019)
- [Vector Search with OpenAI Embeddings: Lucene Is All You Need](https://mlsanity.com/paper/2308.14963/) (2023)
- [Integrating UMLS Knowledge into Large Language Models for Medical Question Answering](https://mlsanity.com/paper/2310.02778/) (2023)
- [LeafAI: query generator for clinical cohort discovery rivaling a human programmer](https://mlsanity.com/paper/2304.06203/) (2023)
- [Sequence Learning with RNNs for Medical Concept Normalization in User-Generated Texts](https://mlsanity.com/paper/1811.11523/) (2018)
- [LinkTransformer: A Unified Package for Record Linkage with Transformer Language Models](https://mlsanity.com/paper/2309.00789/) (2023)
- [MapperGPT: Large Language Models for Linking and Mapping Entities](https://mlsanity.com/paper/2310.03666/) (2023)

## Built on by (most cited first, 7 of 7)

- [High-Throughput Phenotyping of Clinical Text Using Large Language Models](https://mlsanity.com/paper/2408.01214/) (2024)
- [Predicting Failures of LLMs to Link Biomedical Ontology Terms to Identifiers Evidence Across Models and Ontologies](https://mlsanity.com/paper/2509.04458/) (2025)
- [Large Language Model-Based Agents for Automated Research Reproducibility: An Exploratory Study in Alzheimer's Disease](https://mlsanity.com/paper/2505.23852/) (2025)
- [Automated Standardization of Legacy Biomedical Metadata Using an Ontology-Constrained LLM Agent](https://mlsanity.com/paper/2604.08552/) (2026)
- [BeLink: Biomedical Entity Linking Meets Generative Re-Ranking](https://mlsanity.com/paper/2605.22501/) (2026)
- [CUICurate: A GraphRAG-based Framework for Automated Clinical Concept Curation for NLP applications](https://mlsanity.com/paper/2602.17949/) (2026)
- [BELXTR: Biomedical Entity Linking via Contextualized Token Retrieval](https://mlsanity.com/paper/2609.25859/) (2026)

## Abstract (arXiv)

Background: Biomedical entity normalization is critical to biomedical research because the richness of free-text clinical data, such as progress notes, can often be fully leveraged only after translating words and phrases into structured and coded representations suitable for analysis. Large Language Models (LLMs), in turn, have shown great potential and high performance in a variety of natural language processing (NLP) tasks, but their application for normalization remains understudied. Methods: We applied both proprietary and open-source LLMs in combination with several rule-based normalization systems commonly used in biomedical research. We used a two-step LLM integration approach, (1) using an LLM to generate alternative phrasings of a source utterance, and (2) to prune candidate UMLS concepts, using a variety of prompting methods. We measure results by Fβ, where we favor recall over precision, and F1. Results: We evaluated a total of 5,523 concept terms and text contexts from a publicly available dataset of human-annotated biomedical abstracts. Incorporating GPT-3.5-turbo increased overall Fβ and F1 in normalization systems +9.5 and +7.3 (MetaMapLite), +13.9 and +10.9 (QuickUMLS), and +10.5 and +10.3 (BM25), while the open-source Vicuna model achieved +10.8 and +12.2 (MetaMapLite), +14.7 and +15 (QuickUMLS), and +15.6 and +18.7 (BM25). Conclusions: Existing general-purpose LLMs, both propriety and open-source, can be leveraged at scale to greatly improve normalization performance using existing tools, with no fine-tuning.