# Automated Generation and Tagging of Knowledge Components from Multiple-Choice Questions

arXiv: https://arxiv.org/abs/2405.20526 · HTML: https://mlsanity.com/paper/2405.20526/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 49 references that are in the map.
- First posted on arXiv 30 May 2024.

## Builds on (most cited first, 3 of 3)

- [Large Language Model Guided Tree-of-Thought](https://mlsanity.com/paper/2305.08291/) (2023)
- [Ask Me Anything: A simple strategy for prompting language models](https://mlsanity.com/paper/2210.02441/) (2022)
- [Learning Skill Equivalencies Across Platform Taxonomies](https://mlsanity.com/paper/2102.09377/) (2021)

## Built on by (most cited first, 13 of 13)

- [Automated Knowledge Concept Annotation and Question Representation Learning for Knowledge Tracing](https://mlsanity.com/paper/2410.01727/) (2024)
- [KCluster: An LLM-based Clustering Approach to Knowledge Component Discovery](https://mlsanity.com/paper/2505.06469/) (2025)
- [Personalized Exercise Recommendation with Semantically-Grounded Knowledge Tracing](https://mlsanity.com/paper/2507.11060/) (2025)
- [Pattern-based Knowledge Component Extraction from Student Code Using Representation Learning](https://mlsanity.com/paper/2508.09281/) (2025)
- [FoundationalASSIST: An Educational Dataset for Foundational Knowledge Tracing and Pedagogical Grounding of LLMs](https://mlsanity.com/paper/2602.00070/) (2026)
- [Text-Based Approaches to Item Alignment to Content Standards in Large-Scale Reading & Writing Tests](https://mlsanity.com/paper/2509.26431/) (2025)
- [Small but Significant: On the Promise of Small Language Models for Accessible AIED](https://mlsanity.com/paper/2505.08588/) (2025)
- [Using LLMs for Knowledge Component-level Correctness Labeling in Open-ended Coding Problems](https://mlsanity.com/paper/2602.17542/) (2026)
- [Personalized AI Practice Replicates Learning Rate Regularity at Scale](https://mlsanity.com/paper/2604.03246/) (2026)
- [RoMathExam: A Longitudinal Dataset of Romanian Math Exams (1895-2025) with a Seven-Decade Core (1957-2025)](https://mlsanity.com/paper/2604.16392/) (2026)
- [Scalable LLM-based Coding of Dialogue in Healthcare Simulation: Balancing Coding Performance, Processing Time, and Environmental Impact](https://mlsanity.com/paper/2604.23255/) (2026)
- [From Learning Resources to Competencies: LLM-Based Tagging with Evidence and Graph Constraints](https://mlsanity.com/paper/2605.28483/) (2026)
- [SRJudge: Empowering Large Language Models with Selective Reasoning for Fine-Grained Knowledge Concept Tagging](https://mlsanity.com/paper/2609.36982/) (2026)

## Abstract (arXiv)

Knowledge Components (KCs) linked to assessments enhance the measurement of student learning, enrich analytics, and facilitate adaptivity. However, generating and linking KCs to assessment items requires significant effort and domain-specific knowledge. To streamline this process for higher-education courses, we employed GPT-4 to generate KCs for multiple-choice questions (MCQs) in Chemistry and E-Learning. We analyzed discrepancies between the KCs generated by the Large Language Model (LLM) and those made by humans through evaluation from three domain experts in each subject area. This evaluation aimed to determine whether, in instances of non-matching KCs, evaluators showed a preference for the LLM-generated KCs over their human-created counterparts. We also developed an ontology induction algorithm to cluster questions that assess similar KCs based on their content. Our most effective LLM strategy accurately matched KCs for 56% of Chemistry and 35% of E-Learning MCQs, with even higher success when considering the top five KC suggestions. Human evaluators favored LLM-generated KCs, choosing them over human-assigned ones approximately two-thirds of the time, a preference that was statistically significant across both domains. Our clustering algorithm successfully grouped questions by their underlying KCs without needing explicit labels or contextual information. This research advances the automation of KC generation and classification for assessment items, alleviating the need for student data or predefined KC labels.