IDEA2: Expert-in-the-loop competency question elicitation for collaborative ontology engineering
Not on the map. Why?
Abstract (arXiv)
Competency question (CQ) elicitation represents a critical but resource-intensive bottleneck in ontology engineering. This foundational phase is often hampered by the communication gap between domain experts, who possess the necessary knowledge, and ontology engineers, who formalise it. This paper introduces IDEA2, a novel, semi-automated workflow that integrates Large Language Models (LLMs) within a collaborative, expert-in-the-loop process to address this challenge. The methodology is characterised by a core iterative loop: an initial LLM-based extraction of CQs from requirement documents, a co-creational review and feedback phase by domain experts on an accessible collaborative platform, and an iterative, feedback-driven reformulation of rejected CQs by an LLM until consensus is achieved. To ensure transparency and reproducibility, the entire lifecycle of each CQ is tracked using a provenance model that captures the full lineage of edits, anonymised feedback, and generation parameters. The workflow was validated in 2 real-world scenarios (scientific data, cultural heritage), demonstrating that IDEA2 can accelerate the requirements engineering process, improve the acceptance and relevance of the resulting CQs, and exhibit high usability and effectiveness among domain experts. We release all code and experiments at https://github.com/KE-UniLiv/IDEA2
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 13 of its 51 references that are in the map. First posted on arXiv 1 Apr 2026.
Builds on
13 of 51 references.
References · 13
- 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
- 2021A Survey of Human-in-the-loop for Machine Learningcited by 214
- 2024Croissant: A Metadata Format for ML-Ready Datasetscited by 79
- 2023LLMs4OL: Large Language Models for Ontology Learningcited by 75
- 2018CrowdTruth 2.0: Quality Metrics for Crowdsourcing with Disagreementcited by 27
- 2019ArCo: the Italian Cultural Heritage Knowledge Graphcited by 19
- 2024OntoChat: a Framework for Conversational Ontology Engineering using Language Modelscited by 14
- 2023An Experiment in Retrofitting Competency Questions for Existing Ontologiescited by 12
- 2025Large Language Models (LLMs) for Requirements Engineering (RE): A Systematic Literature Reviewcited by 9
- 2024A RAG Approach for Generating Competency Questions in Ontology Engineeringcited by 7
- 2023Using Large Language Models for Knowledge Engineering (LLMKE): A Case Study on Wikidatacited by 5
- 2025Bench4KE: Benchmarking Automated Competency Question Generationcited by 3
Built on by
Not cited yet.