# Low Resource Recognition and Linking of Biomedical Concepts from a Large Ontology

arXiv: https://arxiv.org/abs/2101.10587 · HTML: https://mlsanity.com/paper/2101.10587/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 43 references that are in the map.
- First posted on arXiv 26 Jan 2021.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016)
- [Bidirectional LSTM-CRF Models for Sequence Tagging](https://mlsanity.com/paper/1508.01991/) (2015)
- [ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing](https://mlsanity.com/paper/1902.07669/) (2019)
- [CORD-19: The COVID-19 Open Research Dataset](https://mlsanity.com/paper/2004.10706/) (2020)
- [Zero-Shot Entity Linking by Reading Entity Descriptions](https://mlsanity.com/paper/1906.07348/) (2019)
- [End-to-End Neural Entity Linking](https://mlsanity.com/paper/1808.07699/) (2018)
- [MedMentions: A Large Biomedical Corpus Annotated with UMLS Concepts](https://mlsanity.com/paper/1902.09476/) (2019)
- [DeepType: Multilingual Entity Linking by Neural Type System Evolution](https://mlsanity.com/paper/1802.01021/) (2018)
- [Results of the seventh edition of the BioASQ Challenge](https://mlsanity.com/paper/2006.09174/) (2020)
- [Hierarchical Losses and New Resources for Fine-grained Entity Typing and Linking](https://mlsanity.com/paper/1807.05127/) (2018)
- [Capturing Semantic Similarity for Entity Linking with Convolutional Neural Networks](https://mlsanity.com/paper/1604.00734/) (2016)
- [LATTE: Latent Type Modeling for Biomedical Entity Linking](https://mlsanity.com/paper/1911.09787/) (2019)
- [Effective Use of Bidirectional Language Modeling for Transfer Learning in Biomedical Named Entity Recognition](https://mlsanity.com/paper/1711.07908/) (2017)
- [Linking Entities to Unseen Knowledge Bases with Arbitrary Schemas](https://mlsanity.com/paper/2010.11333/) (2020)
- [Finer Grained Entity Typing with TypeNet](https://mlsanity.com/paper/1711.05795/) (2017)
- [Extracting UMLS Concepts from Medical Text Using General and Domain-Specific Deep Learning Models](https://mlsanity.com/paper/1910.01274/) (2019)

## Built on by (most cited first, 3 of 3)

- [Pretrained Domain-Specific Language Model for General Information Retrieval Tasks in the AEC Domain](https://mlsanity.com/paper/2203.04729/) (2022)
- [Combining Machine Learning and Ontology: A Systematic Literature Review](https://mlsanity.com/paper/2401.07744/) (2024)
- [Class Granularity: How richly does your knowledge graph represent the real world?](https://mlsanity.com/paper/2411.06385/) (2024)

## Abstract (arXiv)

Tools to explore scientific literature are essential for scientists, especially in biomedicine, where about a million new papers are published every year. Many such tools provide users the ability to search for specific entities (e.g. proteins, diseases) by tracking their mentions in papers. PubMed, the most well known database of biomedical papers, relies on human curators to add these annotations. This can take several weeks for new papers, and not all papers get tagged. Machine learning models have been developed to facilitate the semantic indexing of scientific papers. However their performance on the more comprehensive ontologies of biomedical concepts does not reach the levels of typical entity recognition problems studied in NLP. In large part this is due to their low resources, where the ontologies are large, there is a lack of descriptive text defining most entities, and labeled data can only cover a small portion of the ontology. In this paper, we develop a new model that overcomes these challenges by (1) generalizing to entities unseen at training time, and (2) incorporating linking predictions into the mention segmentation decisions. Our approach achieves new state-of-the-art results for the UMLS ontology in both traditional recognition/linking (+8 F1 pts) as well as semantic indexing-based evaluation (+10 F1 pts).