# Benchmarking Resource-Efficient LLMs for Research Topic Ontology Generation in the Biomedical Field

arXiv: https://arxiv.org/abs/2607.17902 · HTML: https://mlsanity.com/paper/2607.17902/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 46 references that are in the map.
- First posted on arXiv 20 Jul 2026.

## Builds on (most cited first, 10 of 10)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [LLMs4OL: Large Language Models for Ontology Learning](https://mlsanity.com/paper/2307.16648/) (2023)
- [Artificial Intelligence for Literature Reviews: Opportunities and Challenges](https://mlsanity.com/paper/2402.08565/) (2024)
- [Ontology Generation using Large Language Models](https://mlsanity.com/paper/2503.05388/) (2025)
- [Constructing Taxonomies from Pretrained Language Models](https://mlsanity.com/paper/2010.12813/) (2020)
- [Are Large Language Models a Good Replacement of Taxonomies?](https://mlsanity.com/paper/2406.11131/) (2024)
- [Large Language Models for Scholarly Ontology Generation: An Extensive Analysis in the Engineering Field](https://mlsanity.com/paper/2412.08258/) (2024)
- [A Survey on Knowledge Organization Systems of Research Fields: Resources and Challenges](https://mlsanity.com/paper/2409.04432/) (2024)

## Abstract (arXiv)

Knowledge Organization Systems like Ontologies and taxonomies are fundamental for structuring scientific knowledge, yet their manual curation presents a persistent bottleneck in knowledge management. While Large Language Models (LLMs) offer a scalable mechanism for automated ontology generation, their capacity to classify complex, domain-specific semantics requires systematic evaluation. In this paper, we assess the performance of five small, open-source LLMs (up to 9 billion parameters) in identifying semantic relationships between biomedical concepts. To support this evaluation, we introduce MeSH-Rel-4K, a dataset comprising 4K semantic relationships extracted from the Medical Subject Headings (MeSH). We analyse three adaptation strategies: standard prompting, Chain-of-Thought prompting, and fine-tuning. While parameter-constrained models traditionally struggle with the nuances of in-context logic, our results reveal that targeted fine-tuning increases the average F1-score by 34.1 percentage points. These results confirm that direct fine-tuning effectively exceeds the reasoning bottlenecks of smaller LLMs, providing an accurate, automated methodology for the construction and evolution of specialised biomedical ontologies.