# The Use of AI Tools to Develop and Validate Q-Matrices

arXiv: https://arxiv.org/abs/2602.08796 · HTML: https://mlsanity.com/paper/2602.08796/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 6 references that are in the map.
- First posted on arXiv 9 Feb 2026.

## Builds on (most cited first, 1 of 1)

- [The Rise of Artificial Intelligence in Educational Measurement: Opportunities and Ethical Challenges](https://mlsanity.com/paper/2406.18900/) (2024)

## Built on by (most cited first, 1 of 1)

- [Curriculum Brain: Constructing Curriculum Knowledge Graphs as a Substrate for Cognitive Diagnosis](https://mlsanity.com/paper/2610.05860/) (2026)

## Abstract (arXiv)

Constructing a Q-matrix is a critical but labor-intensive step in cognitive diagnostic modeling (CDM). This study investigates whether AI tools (i.e., general language models) can support Q-matrix development by comparing AI-generated Q-matrices with a validated Q-matrix from Li and Suen (2013) for a reading comprehension test. In May 2025, multiple AI models were provided with the same training materials as human experts. Agreement among AI-generated Q-matrices, the validated Q-matrix, and human raters' Q-matrices was assessed using Cohen's kappa. Results showed substantial variation across AI models, with Google Gemini 2.5 Pro achieving the highest agreement (Kappa = 0.63) with the validated Q-matrix, exceeding that of all human experts. A follow-up analysis in January 2026 using newer AI versions, however, revealed lower agreement with the validated Q-matrix. Implications and directions for future research are discussed.