# CodeGENCAT: Generative Computerized Adaptive Testing for Open-ended Coding Problems

arXiv: https://arxiv.org/abs/2602.20020 · HTML: https://mlsanity.com/paper/2602.20020/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 30 references that are in the map.
- First posted on arXiv 23 Feb 2026.

## Builds on (most cited first, 9 of 9)

- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [CodeBLEU: a Method for Automatic Evaluation of Code Synthesis](https://mlsanity.com/paper/2009.10297/) (2020)
- [Exploring Knowledge Tracing in Tutor-Student Dialogues using LLMs](https://mlsanity.com/paper/2409.16490/) (2024)
- [SMART: Simulated Students Aligned with Item Response Theory for Question Difficulty Prediction](https://mlsanity.com/paper/2507.05129/) (2025)
- [BOBCAT: Bilevel Optimization-Based Computerized Adaptive Testing](https://mlsanity.com/paper/2108.07386/) (2021)
- [Code-DKT: A Code-based Knowledge Tracing Model for Programming Tasks](https://mlsanity.com/paper/2206.03545/) (2022)
- [Test Case-Informed Knowledge Tracing for Open-ended Coding Tasks](https://mlsanity.com/paper/2410.10829/) (2024)
- [KASER: Knowledge-Aligned Student Error Simulator for Open-Ended Coding Tasks](https://mlsanity.com/paper/2601.06633/) (2026)

## Abstract (arXiv)

Existing Computerized Adaptive Testing (CAT) frameworks typically select questions based on the predicted likelihood that the student will answer correctly. This design ignores information contained in students' open-ended responses, especially in domains such as programming education, where code structures and bugs contain rich information on student knowledge. In this work, we propose Code GENerative CAT (CodeGENCAT), a generative CAT framework that selects questions using predicted student code responses. First, we develop a Generative Item Response Theory (GIRT) model that generates code responses conditioned on estimated student knowledge, trained with supervised fine-tuning followed by direct preference optimization for knowledge-response alignment. Second, we introduce three question-selection algorithms that measure uncertainty, coding style diversity, and information from predicted student code responses. Experiments on two real-world programming education datasets show that CodeGENCAT outperforms all CAT baselines, achieving an AUC improvement of up to 4.32% over the strongest baseline in the early stages of adaptive testing.