# A Language Model based Framework for New Concept Placement in Ontologies

arXiv: https://arxiv.org/abs/2402.17897 · HTML: https://mlsanity.com/paper/2402.17897/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 34 references that are in the map.
- First posted on arXiv 27 Feb 2024.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [Self-Alignment Pretraining for Biomedical Entity Representations](https://mlsanity.com/paper/2010.11784/) (2020)
- [Contextual Semantic Embeddings for Ontology Subsumption Prediction](https://mlsanity.com/paper/2202.09791/) (2022)
- [Exploring Large Language Models for Ontology Alignment](https://mlsanity.com/paper/2309.07172/) (2023)
- [Language Model Analysis for Ontology Subsumption Inference](https://mlsanity.com/paper/2302.06761/) (2023)
- [Taxonomy Completion via Triplet Matching Network](https://mlsanity.com/paper/2101.01896/) (2021)
- [Evaluating Language Models for Knowledge Base Completion](https://mlsanity.com/paper/2303.11082/) (2023)
- [DeepOnto: A Python Package for Ontology Engineering with Deep Learning](https://mlsanity.com/paper/2307.03067/) (2023)
- [Enhancing Taxonomy Completion with Concept Generation via Fusing Relational Representations](https://mlsanity.com/paper/2106.02974/) (2021)
- [Exploring the In-context Learning Ability of Large Language Model for Biomedical Concept Linking](https://mlsanity.com/paper/2307.01137/) (2023)
- [Knowledge Graphs for the Life Sciences: Recent Developments, Challenges and Opportunities](https://mlsanity.com/paper/2309.17255/) (2023)
- [Reveal the Unknown: Out-of-Knowledge-Base Mention Discovery with Entity Linking](https://mlsanity.com/paper/2302.07189/) (2023)
- [Ontology Enrichment from Texts: A Biomedical Dataset for Concept Discovery and Placement](https://mlsanity.com/paper/2306.14704/) (2023)

## Built on by (most cited first, 3 of 3)

- [Automate Knowledge Concept Tagging on Math Questions with LLMs](https://mlsanity.com/paper/2403.17281/) (2024)
- [Hierarchical Retrieval with Out-Of-Vocabulary Queries: A Case Study on SNOMED CT](https://mlsanity.com/paper/2511.16698/) (2025)
- [COntExt: Towards Context-Aware Ontology Extension from Operational Metrics](https://mlsanity.com/paper/2607.29553/) (2026)

## Abstract (arXiv)

We investigate the task of inserting new concepts extracted from texts into an ontology using language models. We explore an approach with three steps: edge search which is to find a set of candidate locations to insert (i.e., subsumptions between concepts), edge formation and enrichment which leverages the ontological structure to produce and enhance the edge candidates, and edge selection which eventually locates the edge to be placed into. In all steps, we propose to leverage neural methods, where we apply embedding-based methods and contrastive learning with Pre-trained Language Models (PLMs) such as BERT for edge search, and adapt a BERT fine-tuning-based multi-label Edge-Cross-encoder, and Large Language Models (LLMs) such as GPT series, FLAN-T5, and Llama 2, for edge selection. We evaluate the methods on recent datasets created using the SNOMED CT ontology and the MedMentions entity linking benchmark. The best settings in our framework use fine-tuned PLM for search and a multi-label Cross-encoder for selection. Zero-shot prompting of LLMs is still not adequate for the task, and we propose explainable instruction tuning of LLMs for improved performance. Our study shows the advantages of PLMs and highlights the encouraging performance of LLMs that motivates future studies.