# Programming Knowledge Tracing: A Comprehensive Dataset and A New Model

arXiv: https://arxiv.org/abs/2112.08273 · HTML: https://mlsanity.com/paper/2112.08273/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 53 references that are in the map.
- First posted on arXiv 11 Dec 2021.

## Builds on (most cited first, 30 of 31)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation](https://mlsanity.com/paper/2102.04664/) (2021)
- [GraphCodeBERT: Pre-training Code Representations with Data Flow](https://mlsanity.com/paper/2009.08366/) (2020)
- [Unified Pre-training for Program Understanding and Generation](https://mlsanity.com/paper/2103.06333/) (2021)
- [Large-Scale Evolution of Image Classifiers](https://mlsanity.com/paper/1703.01041/) (2017)
- [Deep Knowledge Tracing](https://mlsanity.com/paper/1506.05908/) (2015)
- [Revisiting Pre-Trained Models for Chinese Natural Language Processing](https://mlsanity.com/paper/2004.13922/) (2020)
- [code2vec: Learning Distributed Representations of Code](https://mlsanity.com/paper/1803.09473/) (2018)
- [IntelliCode Compose: Code Generation Using Transformer](https://mlsanity.com/paper/2005.08025/) (2020)
- [Context-Aware Attentive Knowledge Tracing](https://mlsanity.com/paper/2007.12324/) (2020)
- [A Self-Attentive model for Knowledge Tracing](https://mlsanity.com/paper/1907.06837/) (2019)
- [Dynamic Key-Value Memory Networks for Knowledge Tracing](https://mlsanity.com/paper/1611.08108/) (2016)
- [Recurrent Neural Network for Text Classification with Multi-Task Learning](https://mlsanity.com/paper/1605.05101/) (2016)
- [Learning to Execute](https://mlsanity.com/paper/1410.4615/) (2014)
- [Retrieval-Augmented Generation for Code Summarization via Hybrid GNN](https://mlsanity.com/paper/2006.05405/) (2020)
- [Language-Agnostic Representation Learning of Source Code from Structure and Context](https://mlsanity.com/paper/2103.11318/) (2021)
- [Addressing Two Problems in Deep Knowledge Tracing via Prediction-Consistent Regularization](https://mlsanity.com/paper/1806.02180/) (2018)
- [GIKT: A Graph-based Interaction Model for Knowledge Tracing](https://mlsanity.com/paper/2009.05991/) (2020)
- [Knowledge Tracing Machines: Factorization Machines for Knowledge Tracing](https://mlsanity.com/paper/1811.03388/) (2018)
- [Improving Knowledge Tracing via Pre-training Question Embeddings](https://mlsanity.com/paper/2012.05031/) (2020)
- [Semantic Sentence Matching with Densely-connected Recurrent and Co-attentive Information](https://mlsanity.com/paper/1805.11360/) (2018)
- [A deep language model for software code](https://mlsanity.com/paper/1608.02715/) (2016)

## Built on by (most cited first, 9 of 9)

- [Test Case-Informed Knowledge Tracing for Open-ended Coding Tasks](https://mlsanity.com/paper/2410.10829/) (2024)
- [CoderAgent: Simulating Student Behavior for Personalized Programming Learning with Large Language Models](https://mlsanity.com/paper/2505.20642/) (2025)
- [GPT-based Open-Ended Knowledge Tracing](https://mlsanity.com/paper/2203.03716/) (2022)
- [Interpreting Latent Student Knowledge Representations in Programming Assignments](https://mlsanity.com/paper/2405.08213/) (2024)
- [CIKT: A Collaborative and Iterative Knowledge Tracing Framework with Large Language Models](https://mlsanity.com/paper/2505.17705/) (2025)
- [MERIT: Memory-Enhanced Retrieval for Interpretable Knowledge Tracing](https://mlsanity.com/paper/2603.22289/) (2026)
- [Personalized Programming Guidance based on Deep Programming Learning Style Capturing](https://mlsanity.com/paper/2403.14638/) (2024)
- [Denoising Programming Knowledge Tracing with a Code Graph-based Tuning Adaptor](https://mlsanity.com/paper/2506.11107/) (2025)
- [Using LLMs for Knowledge Component-level Correctness Labeling in Open-ended Coding Problems](https://mlsanity.com/paper/2602.17542/) (2026)

## Abstract (arXiv)

In this paper, we study knowledge tracing in the domain of programming education and make two important contributions. First, we harvest and publish so far the most comprehensive dataset, namely BePKT, which covers various online behaviors in an OJ system, including programming text problems, knowledge annotations, user-submitted code and system-logged events. Second, we propose a new model PDKT to exploit the enriched context for accurate student behavior prediction. More specifically, we construct a bipartite graph for programming problem embedding, and design an improved pre-training model PLCodeBERT for code embedding, as well as a double-sequence RNN model with exponential decay attention for effective feature fusion. Experimental results on the new dataset BePKT show that our proposed model establishes state-of-the-art performance in programming knowledge tracing. In addition, we verify that our code embedding strategy based on PLCodeBERT is complementary to existing knowledge tracing models to further enhance their accuracy. As a side product, PLCodeBERT also results in better performance in other programming-related tasks such as code clone detection.