# Improving Knowledge Tracing via Pre-training Question Embeddings

arXiv: https://arxiv.org/abs/2012.05031 · HTML: https://mlsanity.com/paper/2012.05031/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Dec 2020.

## Built on by (most cited first, 30 of 36)

- [Knowledge Tracing: A Survey](https://mlsanity.com/paper/2201.06953/) (2022)
- [pyKT: A Python Library to Benchmark Deep Learning based Knowledge Tracing Models](https://mlsanity.com/paper/2206.11460/) (2022)
- [simpleKT: A Simple But Tough-to-Beat Baseline for Knowledge Tracing](https://mlsanity.com/paper/2302.06881/) (2023)
- [Improving Interpretability of Deep Sequential Knowledge Tracing Models with Question-centric Cognitive Representations](https://mlsanity.com/paper/2302.06885/) (2023)
- [Enhancing Deep Knowledge Tracing with Auxiliary Tasks](https://mlsanity.com/paper/2302.07942/) (2023)
- [Bi-CLKT: Bi-Graph Contrastive Learning based Knowledge Tracing](https://mlsanity.com/paper/2201.09020/) (2022)
- [Language Model Can Do Knowledge Tracing: Simple but Effective Method to Integrate Language Model and Knowledge Tracing Task](https://mlsanity.com/paper/2406.02893/) (2024)
- [CLST: Cold-Start Mitigation in Knowledge Tracing by Aligning a Generative Language Model as a Students' Knowledge Tracer](https://mlsanity.com/paper/2406.10296/) (2024)
- [Multi-Factors Aware Dual-Attentional Knowledge Tracing](https://mlsanity.com/paper/2108.04741/) (2021)
- [Programming Knowledge Tracing: A Comprehensive Dataset and A New Model](https://mlsanity.com/paper/2112.08273/) (2021)
- [DGEKT: A Dual Graph Ensemble Learning Method for Knowledge Tracing](https://mlsanity.com/paper/2211.12881/) (2022)
- [Contrastive Cross-Course Knowledge Tracing via Concept Graph Guided Knowledge Transfer](https://mlsanity.com/paper/2505.13489/) (2025)
- [Counterfactual Monotonic Knowledge Tracing for Assessing Students' Dynamic Mastery of Knowledge Concepts](https://mlsanity.com/paper/2308.03377/) (2023)
- [Do We Fully Understand Students' Knowledge States? Identifying and Mitigating Answer Bias in Knowledge Tracing](https://mlsanity.com/paper/2308.07779/) (2023)
- [A Review of Data Mining in Personalized Education: Current Trends and Future Prospects](https://mlsanity.com/paper/2402.17236/) (2024)
- [DKT2: Revisiting Applicable and Comprehensive Knowledge Tracing in Large-Scale Data](https://mlsanity.com/paper/2501.14256/) (2025)
- [Explainable Student Performance Prediction With Personalized Attention for Explaining Why A Student Fails](https://mlsanity.com/paper/2110.08268/) (2021)
- [Domain Generalizable Knowledge Tracing via Concept Aggregation and Relation-Based Attention](https://mlsanity.com/paper/2407.02547/) (2024)
- [Behavior-Aware Item Modeling via Dynamic Procedural Solution Representations for Knowledge Tracing](https://mlsanity.com/paper/2604.08260/) (2026)
- [Quiz-based Knowledge Tracing](https://mlsanity.com/paper/2304.02413/) (2023)
- [Leveraging Skill-to-Skill Supervision for Knowledge Tracing](https://mlsanity.com/paper/2306.06841/) (2023)
- [Scalable and Equitable Math Problem Solving Strategy Prediction in Big Educational Data](https://mlsanity.com/paper/2308.03892/) (2023)
- [Deep Learning for Educational Data Science](https://mlsanity.com/paper/2404.19675/) (2024)
- [Sparse Binary Representation Learning for Knowledge Tracing](https://mlsanity.com/paper/2501.09893/) (2025)
- [HISE-KT: Synergizing Heterogeneous Information Networks and LLMs for Explainable Knowledge Tracing with Meta-Path Optimization](https://mlsanity.com/paper/2511.15191/) (2025)
- [RAG-KT: Cross-platform Explainable Knowledge Tracing with Multi-view Fusion Retrieval Generation](https://mlsanity.com/paper/2604.10960/) (2026)
- [Prerequisite-driven Q-matrix Refinement for Learner Knowledge Assessment: A Case Study in Online Learning Context](https://mlsanity.com/paper/2208.12642/) (2022)
- [Cognition-Mode Aware Variational Representation Learning Framework for Knowledge Tracing](https://mlsanity.com/paper/2309.01179/) (2023)
- [Analysis of Knowledge Tracing performance on synthesised student data](https://mlsanity.com/paper/2401.16832/) (2024)
- [Temporal Graph Memory Networks For Knowledge Tracing](https://mlsanity.com/paper/2410.01836/) (2024)

## Abstract (arXiv)

Knowledge tracing (KT) defines the task of predicting whether students can correctly answer questions based on their historical response. Although much research has been devoted to exploiting the question information, plentiful advanced information among questions and skills hasn't been well extracted, making it challenging for previous work to perform adequately. In this paper, we demonstrate that large gains on KT can be realized by pre-training embeddings for each question on abundant side information, followed by training deep KT models on the obtained embeddings. To be specific, the side information includes question difficulty and three kinds of relations contained in a bipartite graph between questions and skills. To pre-train the question embeddings, we propose to use product-based neural networks to recover the side information. As a result, adopting the pre-trained embeddings in existing deep KT models significantly outperforms state-of-the-art baselines on three common KT datasets.