# Towards an Appropriate Query, Key, and Value Computation for Knowledge Tracing

arXiv: https://arxiv.org/abs/2002.07033 · HTML: https://mlsanity.com/paper/2002.07033/

## Key facts

- Cited by 81 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Feb 2020.

## Built on by (most cited first, 30 of 81)

- [Knowledge Tracing: A Survey](https://mlsanity.com/paper/2201.06953/) (2022)
- [pyKT: A Python Library to Benchmark Deep Learning based Knowledge Tracing Models](https://mlsanity.com/paper/2206.11460/) (2022)
- [simpleKT: A Simple But Tough-to-Beat Baseline for Knowledge Tracing](https://mlsanity.com/paper/2302.06881/) (2023)
- [Exploring Knowledge Tracing in Tutor-Student Dialogues using LLMs](https://mlsanity.com/paper/2409.16490/) (2024)
- [Improving Interpretability of Deep Sequential Knowledge Tracing Models with Question-centric Cognitive Representations](https://mlsanity.com/paper/2302.06885/) (2023)
- [A Survey of Knowledge Tracing: Models, Variants, and Applications](https://mlsanity.com/paper/2105.15106/) (2021)
- [Enhancing Deep Knowledge Tracing with Auxiliary Tasks](https://mlsanity.com/paper/2302.07942/) (2023)
- [Towards Modeling Learner Performance with Large Language Models](https://mlsanity.com/paper/2403.14661/) (2024)
- [Towards Robust Knowledge Tracing Models via k-Sparse Attention](https://mlsanity.com/paper/2407.17097/) (2024)
- [Code-DKT: A Code-based Knowledge Tracing Model for Programming Tasks](https://mlsanity.com/paper/2206.03545/) (2022)
- [Forgetting-aware Linear Bias for Attentive Knowledge Tracing](https://mlsanity.com/paper/2309.14796/) (2023)
- [Predictive, scalable and interpretable knowledge tracing on structured domains](https://mlsanity.com/paper/2403.13179/) (2024)
- [Automated Knowledge Concept Annotation and Question Representation Learning for Knowledge Tracing](https://mlsanity.com/paper/2410.01727/) (2024)
- [DyGKT: Dynamic Graph Learning for Knowledge Tracing](https://mlsanity.com/paper/2407.20824/) (2024)
- [No Task Left Behind: Multi-Task Learning of Knowledge Tracing and Option Tracing for Better Student Assessment](https://mlsanity.com/paper/2204.14006/) (2022)
- [Explainable Few-shot Knowledge Tracing](https://mlsanity.com/paper/2405.14391/) (2024)
- [A Survey of Explainable Knowledge Tracing](https://mlsanity.com/paper/2403.07279/) (2024)
- [Empirical Evaluation of Deep Learning Models for Knowledge Tracing: Of Hyperparameters and Metrics on Performance and Replicability](https://mlsanity.com/paper/2112.15072/) (2021)
- [Mamba4KT:An Efficient and Effective Mamba-based Knowledge Tracing Model](https://mlsanity.com/paper/2405.16542/) (2024)
- [LSTM-SAKT: LSTM-Encoded SAKT-like Transformer for Knowledge Tracing](https://mlsanity.com/paper/2102.00845/) (2021)
- [Last Query Transformer RNN for knowledge tracing](https://mlsanity.com/paper/2102.05038/) (2021)
- [DBE-KT22: A Knowledge Tracing Dataset Based on Online Student Evaluation](https://mlsanity.com/paper/2208.12651/) (2022)
- [Knowledge Tracing for Complex Problem Solving: Granular Rank-Based Tensor Factorization](https://mlsanity.com/paper/2210.09013/) (2022)
- [Do We Fully Understand Students' Knowledge States? Identifying and Mitigating Answer Bias in Knowledge Tracing](https://mlsanity.com/paper/2308.07779/) (2023)
- [A Question-centric Multi-experts Contrastive Learning Framework for Improving the Accuracy and Interpretability of Deep Sequential Knowledge Tracing Models](https://mlsanity.com/paper/2403.07322/) (2024)
- [Riiid! Answer Correctness Prediction Kaggle Challenge: 4th Place Solution Summary](https://mlsanity.com/paper/2102.04250/) (2021)
- [LANA: Towards Personalized Deep Knowledge Tracing Through Distinguishable Interactive Sequences](https://mlsanity.com/paper/2105.06266/) (2021)
- [A Review of Data Mining in Personalized Education: Current Trends and Future Prospects](https://mlsanity.com/paper/2402.17236/) (2024)
- [Personalized Exercise Recommendation with Semantically-Grounded Knowledge Tracing](https://mlsanity.com/paper/2507.11060/) (2025)
- [MUSE: Multi-Scale Temporal Features Evolution for Knowledge Tracing](https://mlsanity.com/paper/2102.00228/) (2021)

## Abstract (arXiv)

Knowledge tracing, the act of modeling a student's knowledge through learning activities, is an extensively studied problem in the field of computer-aided education. Although models with attention mechanism have outperformed traditional approaches such as Bayesian knowledge tracing and collaborative filtering, they share two limitations. Firstly, the models rely on shallow attention layers and fail to capture complex relations among exercises and responses over time. Secondly, different combinations of queries, keys and values for the self-attention layer for knowledge tracing were not extensively explored. Usual practice of using exercises and interactions (exercise-response pairs) as queries and keys/values respectively lacks empirical support. In this paper, we propose a novel Transformer based model for knowledge tracing, SAINT: Separated Self-AttentIve Neural Knowledge Tracing. SAINT has an encoder-decoder structure where exercise and response embedding sequence separately enter the encoder and the decoder respectively, which allows to stack attention layers multiple times. To the best of our knowledge, this is the first work to suggest an encoder-decoder model for knowledge tracing that applies deep self-attentive layers to exercises and responses separately. The empirical evaluations on a large-scale knowledge tracing dataset show that SAINT achieves the state-of-the-art performance in knowledge tracing with the improvement of AUC by 1.8% compared to the current state-of-the-art models.