# Generalizing over Long Tail Concepts for Medical Term Normalization

arXiv: https://arxiv.org/abs/2210.11947 · HTML: https://mlsanity.com/paper/2210.11947/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 24 references that are in the map.
- First posted on arXiv 21 Oct 2022.

## Builds on (most cited first, 10 of 10)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [Self-Alignment Pretraining for Biomedical Entity Representations](https://mlsanity.com/paper/2010.11784/) (2020)
- [SciFive: a text-to-text transformer model for biomedical literature](https://mlsanity.com/paper/2106.03598/) (2021)
- [Biomedical Entity Representations with Synonym Marginalization](https://mlsanity.com/paper/2005.00239/) (2020)
- [CODER: Knowledge infused cross-lingual medical term embedding for term normalization](https://mlsanity.com/paper/2011.02947/) (2020)
- [Medical Coding with Biomedical Transformer Ensembles and Zero/Few-shot Learning](https://mlsanity.com/paper/2206.02662/) (2022)
- [Sequence Learning with RNNs for Medical Concept Normalization in User-Generated Texts](https://mlsanity.com/paper/1811.11523/) (2018)
- [NADE: A Benchmark for Robust Adverse Drug Events Extraction in Face of Negations](https://mlsanity.com/paper/2109.10080/) (2021)
- [Increasing Adverse Drug Events extraction robustness on social media: case study on negation and speculation](https://mlsanity.com/paper/2209.02812/) (2022)

## Built on by (most cited first, 8 of 8)

- [Blind Judgement: Agent-Based Supreme Court Modelling With GPT](https://mlsanity.com/paper/2301.05327/) (2023)
- [Retrieval augmented text-to-SQL generation for epidemiological question answering using electronic health records](https://mlsanity.com/paper/2403.09226/) (2024)
- [BioLORD-2023: Semantic Textual Representations Fusing LLM and Clinical Knowledge Graph Insights](https://mlsanity.com/paper/2311.16075/) (2023)
- [Generating patient cohorts from electronic health records using two-step retrieval-augmented text-to-SQL generation](https://mlsanity.com/paper/2502.21107/) (2025)
- [Disease Entity Recognition and Normalization is Improved with Large Language Model Derived Synthetic Normalized Mentions](https://mlsanity.com/paper/2410.07951/) (2024)
- [Boosting Adverse Drug Event Normalization on Social Media: General-Purpose Model Initialization and Biomedical Semantic Text Similarity Benefit Zero-Shot Linking in Informal Contexts](https://mlsanity.com/paper/2308.00157/) (2023)
- [From Token to Token Pair: Efficient Prompt Compression for Large Language Models in Clinical Prediction](https://mlsanity.com/paper/2605.11774/) (2026)
- [Leveraging Large Language Models for Rare Disease Named Entity Recognition](https://mlsanity.com/paper/2508.09323/) (2025)

## Abstract (arXiv)

Medical term normalization consists in mapping a piece of text to a large number of output classes. Given the small size of the annotated datasets and the extremely long tail distribution of the concepts, it is of utmost importance to develop models that are capable to generalize to scarce or unseen concepts. An important attribute of most target ontologies is their hierarchical structure. In this paper we introduce a simple and effective learning strategy that leverages such information to enhance the generalizability of both discriminative and generative models. The evaluation shows that the proposed strategy produces state-of-the-art performance on seen concepts and consistent improvements on unseen ones, allowing also for efficient zero-shot knowledge transfer across text typologies and datasets.