# Test Case-Informed Knowledge Tracing for Open-ended Coding Tasks

arXiv: https://arxiv.org/abs/2410.10829 · HTML: https://mlsanity.com/paper/2410.10829/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 56 references that are in the map.
- First posted on arXiv 28 Sep 2024.

## Builds on (most cited first, 24 of 24)

- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [CodeBLEU: a Method for Automatic Evaluation of Code Synthesis](https://mlsanity.com/paper/2009.10297/) (2020)
- [Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment](https://mlsanity.com/paper/1610.08452/) (2016)
- [Deep Knowledge Tracing](https://mlsanity.com/paper/1506.05908/) (2015)
- [code2vec: Learning Distributed Representations of Code](https://mlsanity.com/paper/1803.09473/) (2018)
- [Context-Aware Attentive Knowledge Tracing](https://mlsanity.com/paper/2007.12324/) (2020)
- [A Self-Attentive model for Knowledge Tracing](https://mlsanity.com/paper/1907.06837/) (2019)
- [Dynamic Key-Value Memory Networks for Knowledge Tracing](https://mlsanity.com/paper/1611.08108/) (2016)
- [Automatic Generation of Programming Exercises and Code Explanations using Large Language Models](https://mlsanity.com/paper/2206.11861/) (2022)
- [EKT: Exercise-aware Knowledge Tracing for Student Performance Prediction](https://mlsanity.com/paper/1906.05658/) (2019)
- [Knowledge Tracing: A Survey](https://mlsanity.com/paper/2201.06953/) (2022)
- [SAINT+: Integrating Temporal Features for EdNet Correctness Prediction](https://mlsanity.com/paper/2010.12042/) (2020)
- [GIKT: A Graph-based Interaction Model for Knowledge Tracing](https://mlsanity.com/paper/2009.05991/) (2020)
- [Program Synthesis using Natural Language](https://mlsanity.com/paper/1509.00413/) (2015)
- [Synthesizing Tasks for Block-based Programming](https://mlsanity.com/paper/2006.16913/) (2020)
- [Code-DKT: A Code-based Knowledge Tracing Model for Programming Tasks](https://mlsanity.com/paper/2206.03545/) (2022)
- [Improving Socratic Question Generation using Data Augmentation and Preference Optimization](https://mlsanity.com/paper/2403.00199/) (2024)
- [Programming Knowledge Tracing: A Comprehensive Dataset and A New Model](https://mlsanity.com/paper/2112.08273/) (2021)
- [Toward Semi-Automatic Misconception Discovery Using Code Embeddings](https://mlsanity.com/paper/2103.04448/) (2021)
- [From {Solution Synthesis} to {Student Attempt Synthesis} for Block-Based Visual Programming Tasks](https://mlsanity.com/paper/2205.01265/) (2022)
- [Using Large Language Models for Student-Code Guided Test Case Generation in Computer Science Education](https://mlsanity.com/paper/2402.07081/) (2024)
- [Interpreting Latent Student Knowledge Representations in Programming Assignments](https://mlsanity.com/paper/2405.08213/) (2024)

## Built on by (most cited first, 7 of 7)

- [KASER: Knowledge-Aligned Student Error Simulator for Open-Ended Coding Tasks](https://mlsanity.com/paper/2601.06633/) (2026)
- [Automated Knowledge Component Generation for Interpretable Knowledge Tracing in Coding Problems](https://mlsanity.com/paper/2502.18632/) (2025)
- [Learning Code-Edit Embedding to Model Student Debugging Behavior](https://mlsanity.com/paper/2502.19407/) (2025)
- [Teaching Language Models How to Code Like Learners: Conversational Serialization for Student Simulation](https://mlsanity.com/paper/2604.10720/) (2026)
- [Using LLMs for Knowledge Component-level Correctness Labeling in Open-ended Coding Problems](https://mlsanity.com/paper/2602.17542/) (2026)
- [CodeGENCAT: Generative Computerized Adaptive Testing for Open-ended Coding Problems](https://mlsanity.com/paper/2602.20020/) (2026)
- [A Dataset for Modeling Iterative Problem-Solving](https://mlsanity.com/paper/2609.00940/) (2026)

## Abstract (arXiv)

Open-ended coding tasks, which ask students to construct programs according to certain specifications, are common in computer science education. Student modeling can be challenging since their open-ended nature means that student code can be diverse. Traditional knowledge tracing (KT) models that only analyze response correctness may not fully capture nuances in student knowledge from student code. In this paper, we introduce Test case-Informed Knowledge Tracing for Open-ended Coding (TIKTOC), a framework to simultaneously analyze and predict both open-ended student code and whether the code passes each test case. We augment the existing CodeWorkout dataset with the test cases used for a subset of the open-ended coding questions, and propose a multi-task learning KT method to simultaneously analyze and predict 1) whether a student's code submission passes each test case and 2) the student's open-ended code, using a large language model as the backbone. We quantitatively show that these methods outperform existing KT methods for coding that only use the overall score a code submission receives. We also qualitatively demonstrate how test case information, combined with open-ended code, helps us gain fine-grained insights into student knowledge.