# When CQs Go Wrong: Challenges in CQ Verification with OE-Assist

arXiv: https://arxiv.org/abs/2606.24619 · HTML: https://mlsanity.com/paper/2606.24619/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 9 references that are in the map.
- First posted on arXiv 23 Jun 2026.

## Builds on (most cited first, 5 of 5)

- [Ontology Generation using Large Language Models](https://mlsanity.com/paper/2503.05388/) (2025)
- [Discerning and Characterising Types of Competency Questions for Ontologies](https://mlsanity.com/paper/2412.13688/) (2024)
- [Bench4KE: Benchmarking Automated Competency Question Generation](https://mlsanity.com/paper/2505.24554/) (2025)
- [A Comparative Study of Competency Question Elicitation Methods from Ontology Requirements](https://mlsanity.com/paper/2507.02989/) (2025)
- [Large Language Models Assisting Ontology Evaluation](https://mlsanity.com/paper/2507.14552/) (2025)

## Abstract (arXiv)

Competency Questions (CQs) are the central component of CQ-verification, an established process in which an ontology is evaluated against a set of natural language questions to determine whether the intended purpose of the ontology has been properly modelled. However, CQ-verification is often time-consuming and error-prone, as it requires careful interpretation of linguistic nuances and precise alignment with formal ontology constructs. Ambiguities and complexity in CQs can further complicate this process, leading to inconsistent modelling decisions and verification outcomes. In this paper, we investigate what makes a CQ challenging and possible solutions to enhance the users' performance in the CQ-verification process. We experimented with the data of 19 participants who performed CQ-verification on 20 tasks using an LLM assistant to support ontology evaluation. The results show the necessity of a tool to refine CQs before publishing them to avoid ambiguity or excessive complexity in later phases of the ontology engineering process.