# On the Interpretability of Deep Learning Based Models for Knowledge Tracing

arXiv: https://arxiv.org/abs/2101.11335 · HTML: https://mlsanity.com/paper/2101.11335/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 14 references that are in the map.
- First posted on arXiv 27 Jan 2021.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Deep Knowledge Tracing](https://mlsanity.com/paper/1506.05908/) (2015)
- [Dynamic Key-Value Memory Networks for Knowledge Tracing](https://mlsanity.com/paper/1611.08108/) (2016)
- [Visualizing and Understanding Recurrent Networks](https://mlsanity.com/paper/1506.02078/) (2015)
- [EdNet: A Large-Scale Hierarchical Dataset in Education](https://mlsanity.com/paper/1912.03072/) (2019)
- [Addressing Two Problems in Deep Knowledge Tracing via Prediction-Consistent Regularization](https://mlsanity.com/paper/1806.02180/) (2018)
- [No Training Required: Exploring Random Encoders for Sentence Classification](https://mlsanity.com/paper/1901.10444/) (2019)

## Built on by (most cited first, 3 of 3)

- [Assessing the Performance of Online Students -- New Data, New Approaches, Improved Accuracy](https://mlsanity.com/paper/2109.01753/) (2021)
- [Evaluating Adaptive Personalization of Educational Readings with Simulated Learners](https://mlsanity.com/paper/2604.16744/) (2026)
- [RegKT: Interpretable and Robust Deep Knowledge Tracing With IRT-Regularizer](https://mlsanity.com/paper/2609.21791/) (2026)

## Abstract (arXiv)

Knowledge tracing allows Intelligent Tutoring Systems to infer which topics or skills a student has mastered, thus adjusting curriculum accordingly. Deep Learning based models like Deep Knowledge Tracing (DKT) and Dynamic Key-Value Memory Network (DKVMN) have achieved significant improvements compared with models like Bayesian Knowledge Tracing (BKT) and Performance Factors Analysis (PFA). However, these deep learning based models are not as interpretable as other models because the decision-making process learned by deep neural networks is not wholly understood by the research community. In previous work, we critically examined the DKT model, visualizing and analyzing the behaviors of DKT in high dimensional space. In this work, we extend our original analyses with a much larger dataset and add discussions about the memory states of the DKVMN model. We discover that Deep Knowledge Tracing has some critical pitfalls: 1) instead of tracking each skill through time, DKT is more likely to learn an `ability' model; 2) the recurrent nature of DKT reinforces irrelevant information that it uses during the tracking task; 3) an untrained recurrent network can achieve similar results to a trained DKT model, supporting a conclusion that recurrence relations are not properly learned and, instead, improvements are simply a benefit of projection into a high dimensional, sparse vector space. Based on these observations, we propose improvements and future directions for conducting knowledge tracing research using deep neural network models.