# Predicting Failures of LLMs to Link Biomedical Ontology Terms to Identifiers Evidence Across Models and Ontologies

arXiv: https://arxiv.org/abs/2509.04458 · HTML: https://mlsanity.com/paper/2509.04458/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 37 references that are in the map.
- First posted on arXiv 27 Aug 2025.

## Builds on (most cited first, 11 of 11)

- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Large Language Models Struggle to Learn Long-Tail Knowledge](https://mlsanity.com/paper/2211.08411/) (2022)
- [Neural Natural Language Processing for Unstructured Data in Electronic Health Records: a Review](https://mlsanity.com/paper/2107.02975/) (2021)
- [Identifying and Extracting Rare Disease Phenotypes with Large Language Models](https://mlsanity.com/paper/2306.12656/) (2023)
- [An evaluation of GPT models for phenotype concept recognition](https://mlsanity.com/paper/2309.17169/) (2023)
- [Generalizable and Scalable Multistage Biomedical Concept Normalization Leveraging Large Language Models](https://mlsanity.com/paper/2405.15122/) (2024)
- [ChocoLlama: Lessons Learned From Teaching Llamas Dutch](https://mlsanity.com/paper/2412.07633/) (2024)
- [High Throughput Phenotyping of Physician Notes with Large Language and Hybrid NLP Models](https://mlsanity.com/paper/2403.05920/) (2024)
- [EHRmonize: A Framework for Medical Concept Abstraction from Electronic Health Records using Large Language Models](https://mlsanity.com/paper/2407.00242/) (2024)
- [A Large Language Model Outperforms Other Computational Approaches to the High-Throughput Phenotyping of Physician Notes](https://mlsanity.com/paper/2406.14757/) (2024)
- [A Simplified Retriever to Improve Accuracy of Phenotype Normalizations by Large Language Models](https://mlsanity.com/paper/2409.13744/) (2024)

## Built on by (most cited first, 3 of 3)

- [From Memorization to Generalization: Fine-Tuning Large Language Models for Biomedical Term-to-Identifier Normalization](https://mlsanity.com/paper/2510.19036/) (2025)
- [Automated Standardization of Legacy Biomedical Metadata Using an Ontology-Constrained LLM Agent](https://mlsanity.com/paper/2604.08552/) (2026)
- [Latent Knowledge as a Predictor of Fact Acquisition in Fine-Tuned Large Language Models](https://mlsanity.com/paper/2601.18468/) (2026)

## Abstract (arXiv)

Large language models often perform well on biomedical NLP tasks but may fail to link ontology terms to their correct identifiers. We investigate why these failures occur by analyzing predictions across two major ontologies, Human Phenotype Ontology and Gene Ontology, and two high-performing models, GPT-4o and LLaMa 3.1 405B. We evaluate nine candidate features related to term familiarity, identifier usage, morphology, and ontology structure. Univariate and multivariate analyses show that exposure to ontology identifiers is the strongest predictor of linking success.