# ConceptKT: A Benchmark for Concept-Level Deficiency Prediction in Knowledge Tracing

arXiv: https://arxiv.org/abs/2603.24073 · HTML: https://mlsanity.com/paper/2603.24073/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 39 references that are in the map.
- First posted on arXiv 25 Mar 2026.

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalisms](https://mlsanity.com/paper/1905.13319/) (2019)
- [Deep Knowledge Tracing](https://mlsanity.com/paper/1506.05908/) (2015)
- [Context-Aware Attentive Knowledge Tracing](https://mlsanity.com/paper/2007.12324/) (2020)
- [A Self-Attentive model for Knowledge Tracing](https://mlsanity.com/paper/1907.06837/) (2019)
- [Dynamic Key-Value Memory Networks for Knowledge Tracing](https://mlsanity.com/paper/1611.08108/) (2016)
- [EKT: Exercise-aware Knowledge Tracing for Student Performance Prediction](https://mlsanity.com/paper/1906.05658/) (2019)
- [EdNet: A Large-Scale Hierarchical Dataset in Education](https://mlsanity.com/paper/1912.03072/) (2019)
- [Metacognitive Capabilities of LLMs: An Exploration in Mathematical Problem Solving](https://mlsanity.com/paper/2405.12205/) (2024)
- [Neural Cognitive Diagnosis for Intelligent Education Systems](https://mlsanity.com/paper/1908.08733/) (2019)
- [Instructions and Guide for Diagnostic Questions: The NeurIPS 2020 Education Challenge](https://mlsanity.com/paper/2007.12061/) (2020)
- [RKT : Relation-Aware Self-Attention for Knowledge Tracing](https://mlsanity.com/paper/2008.12736/) (2020)
- [Knowledge Tracing with Sequential Key-Value Memory Networks](https://mlsanity.com/paper/1910.13197/) (2019)
- [Exploring Knowledge Tracing in Tutor-Student Dialogues using LLMs](https://mlsanity.com/paper/2409.16490/) (2024)
- [DrawEduMath: Evaluating Vision Language Models with Expert-Annotated Students' Hand-Drawn Math Images](https://mlsanity.com/paper/2501.14877/) (2025)
- [DyGKT: Dynamic Graph Learning for Knowledge Tracing](https://mlsanity.com/paper/2407.20824/) (2024)
- [KnowTrace: Bootstrapping Iterative Retrieval-Augmented Generation with Structured Knowledge Tracing](https://mlsanity.com/paper/2505.20245/) (2025)
- [LLM-KT: Aligning Large Language Models with Knowledge Tracing using a Plug-and-Play Instruction](https://mlsanity.com/paper/2502.02945/) (2025)
- [Automated Feedback in Math Education: A Comparative Analysis of LLMs for Open-Ended Responses](https://mlsanity.com/paper/2411.08910/) (2024)
- [LLM-driven Effective Knowledge Tracing by Integrating Dual-channel Difficulty](https://mlsanity.com/paper/2502.19915/) (2025)

## Abstract (arXiv)

Knowledge Tracing (KT) is a critical technique for modeling student knowledge to support personalized learning. However, most KT systems focus on binary correctness prediction and cannot diagnose the underlying conceptual misunderstandings that lead to errors. Such fine-grained diagnostic feedback is essential for designing targeted instruction and effective remediation. In this work, we introduce the task of concept-level deficiency prediction, which extends traditional KT by identifying the specific concepts a student is likely to struggle with on future problems. We present ConceptKT, a dataset annotated with labels that capture both the concepts required to solve each question and the missing concepts underlying incorrect responses. We investigate in-context learning approaches to KT and evaluate the diagnostic capabilities of various Large Language Models (LLMs) and Large Reasoning Models (LRMs). Different strategies for selecting informative historical records are explored. Experimental results demonstrate that selecting response histories based on conceptual alignment and semantic similarity leads to improved performance on both correctness prediction and concept-level deficiency identification.