# Distilling Closed-Source LLM's Knowledge for Locally Stable and Economic Biomedical Entity Linking

arXiv: https://arxiv.org/abs/2505.19722 · HTML: https://mlsanity.com/paper/2505.19722/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 35 references that are in the map.
- First posted on arXiv 26 May 2025.

## Builds on (most cited first, 26 of 26)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Efficient and Effective Text Encoding for Chinese LLaMA and Alpaca](https://mlsanity.com/paper/2304.08177/) (2023)
- [Symbolic Knowledge Distillation: from General Language Models to Commonsense Models](https://mlsanity.com/paper/2110.07178/) (2021)
- [Self-Alignment Pretraining for Biomedical Entity Representations](https://mlsanity.com/paper/2010.11784/) (2020)
- [HuaTuo: Tuning LLaMA Model with Chinese Medical Knowledge](https://mlsanity.com/paper/2304.06975/) (2023)
- [Biomedical Entity Representations with Synonym Marginalization](https://mlsanity.com/paper/2005.00239/) (2020)
- [Zero-shot Temporal Relation Extraction with ChatGPT](https://mlsanity.com/paper/2304.05454/) (2023)
- [BERT-based Ranking for Biomedical Entity Normalization](https://mlsanity.com/paper/1908.03548/) (2019)
- [Who Wrote it and Why? Prompting Large-Language Models for Authorship Verification](https://mlsanity.com/paper/2310.08123/) (2023)
- [Graph Soft-Contrastive Learning via Neighborhood Ranking](https://mlsanity.com/paper/2209.13964/) (2022)
- [LightCAKE: A Lightweight Framework for Context-Aware Knowledge Graph Embedding](https://mlsanity.com/paper/2102.10826/) (2021)
- [Resolving the Imbalance Issue in Hierarchical Disciplinary Topic Inference via LLM-based Data Augmentation](https://mlsanity.com/paper/2310.05318/) (2023)
- [scCDCG: Efficient Deep Structural Clustering for single-cell RNA-seq via Deep Cut-informed Graph Embedding](https://mlsanity.com/paper/2404.06167/) (2024)
- [Adaptive Path-Memory Network for Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2304.12604/) (2023)
- [Temporal Inductive Path Neural Network for Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2309.03251/) (2023)
- [FedGCS: A Generative Framework for Efficient Client Selection in Federated Learning via Gradient-based Optimization](https://mlsanity.com/paper/2405.06312/) (2024)
- [NEEDED: Introducing Hierarchical Transformer to Eye Diseases Diagnosis](https://mlsanity.com/paper/2212.13408/) (2022)
- [Deep Cut-informed Graph Embedding and Clustering](https://mlsanity.com/paper/2503.06635/) (2025)
- [Context-Enhanced Entity and Relation Embedding for Knowledge Graph Completion](https://mlsanity.com/paper/2012.07011/) (2020)
- [Rethinking Graph Contrastive Learning through Relative Similarity Preservation](https://mlsanity.com/paper/2505.05533/) (2025)

## Built on by (most cited first, 1 of 1)

- [LongBEL: Long-Context and Document-Consistent Biomedical Entity Linking](https://mlsanity.com/paper/2605.13451/) (2026)

## Abstract (arXiv)

Biomedical entity linking aims to map nonstandard entities to standard entities in a knowledge base. Traditional supervised methods perform well but require extensive annotated data to transfer, limiting their usage in low-resource scenarios. Large language models (LLMs), especially closed-source LLMs, can address these but risk stability issues and high economic costs: using these models is restricted by commercial companies and brings significant economic costs when dealing with large amounts of data. To address this, we propose ``RPDR'', a framework combining closed-source LLMs and open-source LLMs for re-ranking candidates retrieved by a retriever fine-tuned with a small amount of data. By prompting a closed-source LLM to generate training data from unannotated data and fine-tuning an open-source LLM for re-ranking, we effectively distill the knowledge to the open-source LLM that can be deployed locally, thus avoiding the stability issues and the problem of high economic costs. We evaluate RPDR on two datasets, including one real-world dataset and one publicly available dataset involving two languages: Chinese and English. RPDR achieves 0.019 Acc@1 improvement and 0.036 Acc@1 improvement on the Aier dataset and the Ask A Patient dataset when the amount of training data is not enough. The results demonstrate the superiority and generalizability of the proposed framework.