# Towards Robust Knowledge Tracing Models via k-Sparse Attention

arXiv: https://arxiv.org/abs/2407.17097 · HTML: https://mlsanity.com/paper/2407.17097/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 24 references that are in the map.
- First posted on arXiv 24 Jul 2024.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Generating Long Sequences with Sparse Transformers](https://mlsanity.com/paper/1904.10509/) (2019)
- [From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification](https://mlsanity.com/paper/1602.02068/) (2016)
- [Deep Knowledge Tracing](https://mlsanity.com/paper/1506.05908/) (2015)
- [Context-Aware Attentive Knowledge Tracing](https://mlsanity.com/paper/2007.12324/) (2020)
- [A Self-Attentive model for Knowledge Tracing](https://mlsanity.com/paper/1907.06837/) (2019)
- [Dynamic Key-Value Memory Networks for Knowledge Tracing](https://mlsanity.com/paper/1611.08108/) (2016)
- [Sparse Sequence-to-Sequence Models](https://mlsanity.com/paper/1905.05702/) (2019)
- [Towards an Appropriate Query, Key, and Value Computation for Knowledge Tracing](https://mlsanity.com/paper/2002.07033/) (2020)
- [Instructions and Guide for Diagnostic Questions: The NeurIPS 2020 Education Challenge](https://mlsanity.com/paper/2007.12061/) (2020)
- [Explicit Sparse Transformer: Concentrated Attention Through Explicit Selection](https://mlsanity.com/paper/1912.11637/) (2019)
- [RKT : Relation-Aware Self-Attention for Knowledge Tracing](https://mlsanity.com/paper/2008.12736/) (2020)
- [Knowledge Tracing with Sequential Key-Value Memory Networks](https://mlsanity.com/paper/1910.13197/) (2019)
- [Addressing Two Problems in Deep Knowledge Tracing via Prediction-Consistent Regularization](https://mlsanity.com/paper/1806.02180/) (2018)
- [pyKT: A Python Library to Benchmark Deep Learning based Knowledge Tracing Models](https://mlsanity.com/paper/2206.11460/) (2022)
- [simpleKT: A Simple But Tough-to-Beat Baseline for Knowledge Tracing](https://mlsanity.com/paper/2302.06881/) (2023)
- [Improving Interpretability of Deep Sequential Knowledge Tracing Models with Question-centric Cognitive Representations](https://mlsanity.com/paper/2302.06885/) (2023)
- [Enhancing Knowledge Tracing via Adversarial Training](https://mlsanity.com/paper/2108.04430/) (2021)
- [Enhancing Deep Knowledge Tracing with Auxiliary Tasks](https://mlsanity.com/paper/2302.07942/) (2023)
- [Knowledge Query Network: How Knowledge Interacts with Skills](https://mlsanity.com/paper/1908.02146/) (2019)

## Built on by (most cited first, 19 of 19)

- [Disentangled Knowledge Tracing for Alleviating Cognitive Bias](https://mlsanity.com/paper/2503.02539/) (2025)
- [Automated Knowledge Concept Annotation and Question Representation Learning for Knowledge Tracing](https://mlsanity.com/paper/2410.01727/) (2024)
- [Contrastive Cross-Course Knowledge Tracing via Concept Graph Guided Knowledge Transfer](https://mlsanity.com/paper/2505.13489/) (2025)
- [Cold Start Problem: An Experimental Study of Knowledge Tracing Models with New Students](https://mlsanity.com/paper/2505.21517/) (2025)
- [DKT2: Revisiting Applicable and Comprehensive Knowledge Tracing in Large-Scale Data](https://mlsanity.com/paper/2501.14256/) (2025)
- [Personalized Exercise Recommendation with Semantically-Grounded Knowledge Tracing](https://mlsanity.com/paper/2507.11060/) (2025)
- [Who Am I? History-Aware Profiles for Student Simulation in Tutoring Dialogues](https://mlsanity.com/paper/2605.30051/) (2026)
- [Model Merging for Knowledge Editing](https://mlsanity.com/paper/2506.12384/) (2025)
- [Behavior-Aware Item Modeling via Dynamic Procedural Solution Representations for Knowledge Tracing](https://mlsanity.com/paper/2604.08260/) (2026)
- [Interpretable Difficulty-Aware Knowledge Tracing in Tutor-Student Dialogues](https://mlsanity.com/paper/2605.01097/) (2026)
- [LLM-driven Effective Knowledge Tracing by Integrating Dual-channel Difficulty](https://mlsanity.com/paper/2502.19915/) (2025)
- [Personalized Knowledge Tracing through Student Representation Reconstruction and Class Imbalance Mitigation](https://mlsanity.com/paper/2409.06745/) (2024)
- [RouterKT: Mixture-of-Experts for Knowledge Tracing](https://mlsanity.com/paper/2504.08989/) (2025)
- [Advancing Knowledge Tracing by Exploring Follow-up Performance Trends](https://mlsanity.com/paper/2508.08019/) (2025)
- [KeenKT: Knowledge Mastery-State Disambiguation for Knowledge Tracing](https://mlsanity.com/paper/2512.18709/) (2025)
- [Ensuring Reliability in Programming Knowledge Tracing: A Re-evaluation of Attention-augmented Models and Experimental Protocols](https://mlsanity.com/paper/2605.04727/) (2026)
- [Temporal Smoothness Doubly Robust Learning for Debiased Knowledge Tracing](https://mlsanity.com/paper/2605.05958/) (2026)
- [Disentangling Knowledge States with Ability and Proficiency Modeling for Knowledge Tracing](https://mlsanity.com/paper/2607.13103/) (2026)
- [An Information-Theoretic Evaluation Framework for Benchmark and Model Diagnosis in Knowledge Tracing](https://mlsanity.com/paper/2610.06988/) (2026)

## Abstract (arXiv)

Knowledge tracing (KT) is the problem of predicting students' future performance based on their historical interaction sequences. With the advanced capability of capturing contextual long-term dependency, attention mechanism becomes one of the essential components in many deep learning based KT (DLKT) models. In spite of the impressive performance achieved by these attentional DLKT models, many of them are often vulnerable to run the risk of overfitting, especially on small-scale educational datasets. Therefore, in this paper, we propose textscsparseKT, a simple yet effective framework to improve the robustness and generalization of the attention based DLKT approaches. Specifically, we incorporate a k-selection module to only pick items with the highest attention scores. We propose two sparsification heuristics : (1) soft-thresholding sparse attention and (2) top-K sparse attention. We show that our textscsparseKT is able to help attentional KT models get rid of irrelevant student interactions and have comparable predictive performance when compared to 11 state-of-the-art KT models on three publicly available real-world educational datasets. To encourage reproducible research, we make our data and code publicly available at urlhttps://github.com/pykt-team/pykt-toolkitfootnoteWe merged our model to the textscpyKT benchmark at urlhttps://pykt.org/..