# Ontology Generation using Large Language Models

arXiv: https://arxiv.org/abs/2503.05388 · HTML: https://mlsanity.com/paper/2503.05388/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 37 references that are in the map.
- First posted on arXiv 7 Mar 2025.

## Builds on (most cited first, 11 of 11)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Decomposed Prompting: A Modular Approach for Solving Complex Tasks](https://mlsanity.com/paper/2210.02406/) (2022)
- [Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs](https://mlsanity.com/paper/2402.03927/) (2024)
- [Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters](https://mlsanity.com/paper/2212.10001/) (2022)
- [Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Future](https://mlsanity.com/paper/2309.15402/) (2023)
- [LLMs4OL: Large Language Models for Ontology Learning](https://mlsanity.com/paper/2307.16648/) (2023)
- [Metacognitive Prompting Improves Understanding in Large Language Models](https://mlsanity.com/paper/2308.05342/) (2023)
- [Ontology engineering with Large Language Models](https://mlsanity.com/paper/2307.16699/) (2023)
- [LLMs4Life: Large Language Models for Ontology Learning in Life Sciences](https://mlsanity.com/paper/2412.02035/) (2024)
- [A Benchmark for the Detection of Metalinguistic Disagreements between LLMs and Knowledge Graphs](https://mlsanity.com/paper/2502.02896/) (2025)

## Built on by (most cited first, 25 of 25)

- [LLM-empowered knowledge graph construction: A survey](https://mlsanity.com/paper/2510.20345/) (2025)
- [Assessing the Capability of Large Language Models for Domain-Specific Ontology Generation](https://mlsanity.com/paper/2504.17402/) (2025)
- [Towards a Common Framework for Autoformalization](https://mlsanity.com/paper/2509.09810/) (2025)
- [Retrieval-Augmented Generation of Ontologies from Relational Databases](https://mlsanity.com/paper/2506.01232/) (2025)
- [Automated Skill Decomposition Meets Expert Ontologies: Bridging the Granularity Gap with LLMs](https://mlsanity.com/paper/2510.11313/) (2025)
- [Ontology Learning with LLMs: A Benchmark Study on Axiom Identification](https://mlsanity.com/paper/2512.05594/) (2025)
- [Large Language Models Assisting Ontology Evaluation](https://mlsanity.com/paper/2507.14552/) (2025)
- [Investigating Language Model Capabilities to Represent and Process Formal Knowledge: A Preliminary Study to Assist Ontology Engineering](https://mlsanity.com/paper/2509.10249/) (2025)
- [LLM-Driven Ontology Construction for Enterprise Knowledge Graphs](https://mlsanity.com/paper/2602.01276/) (2026)
- [OntoExtend: A Framework for Requirement-driven and Scalable Ontology Extension with LLMs](https://mlsanity.com/paper/2607.17963/) (2026)
- [Towards Automated Ontology Generation from Unstructured Text: A Multi-Agent LLM Approach](https://mlsanity.com/paper/2604.23090/) (2026)
- [Streamlining Knowledge Graph Creation with PyRML](https://mlsanity.com/paper/2505.20949/) (2025)
- [LLM-based Zero-shot Triple Extraction for Automated Ontology Generation from Software Engineering Standards](https://mlsanity.com/paper/2509.00140/) (2025)
- [Meow: End-to-End Outline Writing for Automatic Academic Survey](https://mlsanity.com/paper/2509.19370/) (2025)
- [Automatic Ontology Construction Using LLMs as an External Layer of Memory, Verification, and Planning for Hybrid Intelligent Systems](https://mlsanity.com/paper/2604.20795/) (2026)
- [BLAST: Benchmarking LLMs with ASP-based Structured Testing](https://mlsanity.com/paper/2604.22306/) (2026)
- [Provably Auditable and Safe LLM Agents from Human-Authored Ontologies](https://mlsanity.com/paper/2606.04903/) (2026)
- [When CQs Go Wrong: Challenges in CQ Verification with OE-Assist](https://mlsanity.com/paper/2606.24619/) (2026)
- [JD Oxygen AI Item Center (Oxygen AIIC) V1: An Industrial-Scale LLM/VLM-Centric Solution for Item Understanding, Management, and Applications](https://mlsanity.com/paper/2606.28070/) (2026)
- [Benchmarking Resource-Efficient LLMs for Research Topic Ontology Generation in the Biomedical Field](https://mlsanity.com/paper/2607.17902/) (2026)
- [Reversing Arrows in Large Language Models](https://mlsanity.com/paper/2608.03512/) (2026)
- [When Does Bigger Help? A Controlled Study of LLM Scale for Ontology Learning](https://mlsanity.com/paper/2608.31118/) (2026)
- [CQ4OE: A benchmark for assessing LLM-assisted ontology generation from competency questions](https://mlsanity.com/paper/2609.26029/) (2026)
- [LLM-Guided Ontology-Driven Knowledge Graph Construction from Unstructured Text](https://mlsanity.com/paper/2609.31663/) (2026)
- [DTScribe: Generating Models for Digital Twin Reporting using Local Large Language Models](https://mlsanity.com/paper/2610.18635/) (2026)

## Abstract (arXiv)

The ontology engineering process is complex, time-consuming, and error-prone, even for experienced ontology engineers. In this work, we investigate the potential of Large Language Models (LLMs) to provide effective OWL ontology drafts directly from ontological requirements described using user stories and competency questions. Our main contribution is the presentation and evaluation of two new prompting techniques for automated ontology development: Memoryless CQbyCQ and Ontogenia. We also emphasize the importance of three structural criteria for ontology assessment, alongside expert qualitative evaluation, highlighting the need for a multi-dimensional evaluation in order to capture the quality and usability of the generated ontologies. Our experiments, conducted on a benchmark dataset of ten ontologies with 100 distinct CQs and 29 different user stories, compare the performance of three LLMs using the two prompting techniques. The results demonstrate improvements over the current state-of-the-art in LLM-supported ontology engineering. More specifically, the model OpenAI o1-preview with Ontogenia produces ontologies of sufficient quality to meet the requirements of ontology engineers, significantly outperforming novice ontology engineers in modelling ability. However, we still note some common mistakes and variability of result quality, which is important to take into account when using LLMs for ontology authoring support. We discuss these limitations and propose directions for future research.