Towards Valid Student Simulation with Large Language Models
Not on the map. Why?
Abstract (arXiv)
This paper presents a conceptual and methodological framework for large language model (LLM) based student simulation in educational settings. The authors identify a core failure mode, termed the "competence paradox" in which broadly capable LLMs are asked to emulate partially knowledgeable learners, leading to unrealistic error patterns and learning dynamics. To address this, the paper reframes student simulation as a constrained generation problem governed by an explicit Epistemic State Specification (ESS), which defines what a simulated learner can access, how errors are structured, and how learner state evolves over time. The work further introduces a Goal-by-Environment framework to situate simulated student systems according to behavioral objectives and deployment contexts. Rather than proposing a new system or benchmark, the paper synthesizes prior literature, formalizes key design dimensions, and articulates open challenges related to validity, evaluation, and ethical risks. Overall, the paper argues for epistemic fidelity over surface realism as a prerequisite for using LLM-based simulated students as reliable scientific and pedagogical instruments.
Key facts. Cited by 7 papers in the map (data through 13 Oct 2026). Builds on 26 of its 75 references that are in the map. First posted on arXiv 9 Jan 2026.
Builds on
26 of 75 references.
References · 26
- 2024Simulating Classroom Education with LLM-Empowered Agentscited by 126
- 2025LLM Generated Persona is a Promise with a Catchcited by 79
- 2024Generative Students: Using LLM-Simulated Student Profiles to Support Question Item Evaluationcited by 57
- 2023Teach AI How to Code: Using Large Language Models as Teachable Agents for Programming Educationcited by 39
- 2024MathVC: An LLM-Simulated Multi-Character Virtual Classroom for Mathematics Educationcited by 36
- 2025Agent4Edu: Generating Learner Response Data by Generative Agents for Intelligent Education Systemscited by 32
- 2023How to Teach Programming in the AI Era? Using LLMs as a Teachable Agent for Debuggingcited by 30
- 2024TeachTune: Reviewing Pedagogical Agents Against Diverse Student Profiles with Simulated Studentscited by 25
- 2025SMART: Simulated Students Aligned with Item Response Theory for Question Difficulty Predictioncited by 21
- 2025Learning to Make MISTAKEs: Modeling Incorrect Student Thinking And Key Errorscited by 20
- 2025Embracing Imperfection: Simulating Students with Diverse Cognitive Levels Using LLM-based Agentscited by 19
- 2025SciArena: An Open Evaluation Platform for Non-Verifiable Scientific Literature-Grounded Taskscited by 19
14 more
- 2024LLM-based Cognitive Models of Students with Misconceptionscited by 16
- 2025Hire Your Anthropologist! Rethinking Culture Benchmarks Through an Anthropological Lenscited by 13
- 2025Humanizing Machines: Rethinking LLM Anthropomorphism Through a Multi-Level Framework of Designcited by 11
- 2025Text-Based Approaches to Item Difficulty Modeling in Large-Scale Assessments: A Systematic Reviewcited by 10
- 2025Modeling Student Learning with 3.8 Million Program Tracescited by 10
- 2025Embracing Contradiction: Theoretical Inconsistency Will Not Impede the Road of Building Responsible AI Systemscited by 8
- 2025FACET: Teacher-Centred LLM-Based Multi-Agent Systems-Towards Personalized Educational Worksheetscited by 7
- 2025Can LLMs Estimate Student Struggles? Human-AI Difficulty Alignment with Proficiency Simulation for Item Difficulty Predictioncited by 7
- 2024Privacy Bias in Language Models: A Contextual Integrity-based Auditing Metriccited by 6
- 2024When LLMs Learn to be Students: The SOEI Framework for Modeling and Evaluating Virtual Student Agents in Educational Interactioncited by 4
- 2024Generative Co-Learners: Enhancing Cognitive and Social Presence of Students in Asynchronous Learning with Generative AIcited by 3
- 2025SimulRAG: Simulator-based RAG for Grounding LLMs in Long-form Scientific QAcited by 3
- 2025Building Scaffolding Dialogue Data with LLM-Simulated Novicescited by 2
- 2025Generative Models for Synthetic Data: Transforming Data Mining in the GenAI Eracited by 2
Built on by
Citing papers per year: 2026: 7.
7 papers.
- 2026Cognitive Episodes in LLM Reasoning Traces Enable Interpretable Human Item Difficulty Predictioncited by 2
- 2026BEAGLE: Behavior-Enforced Agent for Grounded Learner Emulationcited by 1
- 2026Rethinking LLM-Judged Helpfulness as a Pedagogy Signal: A Pre-Registered Audit Across Tutor Modelscited by 1
- 2026StudentSim: Training LLM-based Student Simulatorscited by 1
- 2026MCQ Difficulty Prediction via Modeling Learner Heterogeneity Using Data-Driven Cognitive Profilingcited by 0
- 2026Representing Visual Evidence for Item Difficulty Prediction: Visual Textualization and Image-Native Modelingcited by 0
- 2026EvalConvoLearn: An Open-Source Framework for Evaluating Grounded Learner Simulations in Tutoring Conversationscited by 0
- 2026StudentSim: Training LLM-based Student Simulators1 Sep 2026
- 2026Representing Visual Evidence for Item Difficulty Prediction: Visual Textualization and Image-Native Modeling5 Aug 2026
- 2026Rethinking LLM-Judged Helpfulness as a Pedagogy Signal: A Pre-Registered Audit Across Tutor Models30 Jul 2026
- 2026Cognitive Episodes in LLM Reasoning Traces Enable Interpretable Human Item Difficulty Prediction26 Jun 2026
- 2026EvalConvoLearn: An Open-Source Framework for Evaluating Grounded Learner Simulations in Tutoring Conversations18 Jun 2026
- 2026MCQ Difficulty Prediction via Modeling Learner Heterogeneity Using Data-Driven Cognitive Profiling13 Apr 2026
- 2026BEAGLE: Behavior-Enforced Agent for Grounded Learner Emulation6 Feb 2026