# Constructing a Question-Answering Simulator through the Distillation of LLMs

arXiv: https://arxiv.org/abs/2509.09226 · HTML: https://mlsanity.com/paper/2509.09226/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 23 references that are in the map.
- First posted on arXiv 11 Sep 2025.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [GLM: General Language Model Pretraining with Autoregressive Blank Infilling](https://mlsanity.com/paper/2103.10360/) (2021)
- [Deep Knowledge Tracing](https://mlsanity.com/paper/1506.05908/) (2015)
- [Context-Aware Attentive Knowledge Tracing](https://mlsanity.com/paper/2007.12324/) (2020)
- [A Self-Attentive model for Knowledge Tracing](https://mlsanity.com/paper/1907.06837/) (2019)
- [An Empirical Study of the Non-determinism of ChatGPT in Code Generation](https://mlsanity.com/paper/2308.02828/) (2023)
- [Deep-IRT: Make Deep Learning Based Knowledge Tracing Explainable Using Item Response Theory](https://mlsanity.com/paper/1904.11738/) (2019)
- [Agent4Edu: Generating Learner Response Data by Generative Agents for Intelligent Education Systems](https://mlsanity.com/paper/2501.10332/) (2025)
- [Enhancing Knowledge Tracing via Adversarial Training](https://mlsanity.com/paper/2108.04430/) (2021)
- [Disentangled Knowledge Tracing for Alleviating Cognitive Bias](https://mlsanity.com/paper/2503.02539/) (2025)
- [Set-to-Sequence Ranking-based Concept-aware Learning Path Recommendation](https://mlsanity.com/paper/2306.04234/) (2023)
- [RLTutor: Reinforcement Learning Based Adaptive Tutoring System by Modeling Virtual Student with Fewer Interactions](https://mlsanity.com/paper/2108.00268/) (2021)
- [HierLLM: Hierarchical Large Language Model for Question Recommendation](https://mlsanity.com/paper/2409.06177/) (2024)

## Abstract (arXiv)

The question-answering (QA) simulator is a model that mimics real student learning behaviors and predicts their correctness of their responses to questions. QA simulators enable educational recommender systems (ERS) to collect large amounts of training data without interacting with real students, thereby preventing harmful recommendations made by an undertrained ERS from undermining actual student learning. Given the QA history, there are two categories of solutions to predict the correctness, conducting the simulation: (1) LLM-free methods, which apply a traditional sequential model to transfer the QA history into a vector representation first, and make predictions based on the representation; (2) LLM-based methods, which leverage the domain knowledge and reasoning capability of LLM to enhence the prediction. LLM-free methods offer fast inference but generally yield suboptimal performance. In contrast, most LLM-based methods achieve better results, but at the cost of slower inference speed and higher GPU memory consumption. In this paper, we propose a method named LLM Distillation based Simulator (LDSim), which distills domain knowledge and reasoning capability from an LLM to better assist prediction, thereby improving simulation performance. Extensive experiments demonstrate that our LDSim achieves strong results on both the simulation task and the knowledge tracing (KT) task. Our code is publicly available at https://anonymous.4open.science/r/LDSim-05A9.