mlsanityData to 14 Oct 2026

ParaStudent: Closing the Sim2Real Gap in User Simulators for AI Tutor Evaluation

arXiv 2507.12674 ↗First posted 16 Jul 2025cs.CY cs.AI cs.SEcited by 2

Not on the map. Why?

Abstract (arXiv)

Evaluating Artificial Intelligence (AI) tutor feedback before deployment requires anticipating student engagement, typically assessed through real interaction data. We introduce ParaStudent, a fine-tuning framework for simulating novice programming revisions to support AI tutor evaluation. Compared with prompted baselines, ParaStudent's revisions more closely match real student code distributions across functional, stylistic, and semantic metrics. Our best variant achieves AUCs of 0.80 for both feedback relevance and successful uptake when distinguishing streams with real engagement above versus at or below the median, while prompted baselines remain near chance on successful uptake. These findings demonstrate the promise of simulated engagement for pre-deployment feedback triage.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 28 of its 75 references that are in the map. First posted on arXiv 16 Jul 2025.

Builds on

28 of 75 references.

References · 28

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  3. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
  4. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
  5. 2024Simulating Classroom Education with LLM-Empowered Agentscited by 127
  6. 2024CodeAid: Evaluating a Classroom Deployment of an LLM-based Programming Assistant that Balances Student and Educator Needscited by 84
  7. 2025Language Model Fine-Tuning on Scaled Survey Data for Predicting Distributions of Public Opinionscited by 66
  8. 2024Unifying AI Tutor Evaluation: An Evaluation Taxonomy for Pedagogical Ability Assessment of LLM-Powered AI Tutorscited by 61
  9. 2024CoIR: A Comprehensive Benchmark for Code Information Retrieval Modelscited by 60
  10. 2024Generative Students: Using LLM-Simulated Student Profiles to Support Question Item Evaluationcited by 58
  11. 2023Rethinking Model Evaluation as Narrowing the Socio-Technical Gapcited by 56
  12. 2025MathTutorBench: A Benchmark for Measuring Open-ended Pedagogical Capabilities of LLM Tutorscited by 49
16 more
  1. 2023Zero-Shot Goal-Directed Dialogue via RL on Imagined Conversationscited by 38
  2. 2024CodeXEmbed: A Generalist Embedding Model Family for Multiligual and Multi-task Code Retrievalcited by 34
  3. 2025SimUSER: Simulating User Behavior with Large Language Models for Recommender System Evaluationcited by 33
  4. 2025SynthesizeMe! Inducing Persona-Guided Prompts for Personalized Reward Models in LLMscited by 29
  5. 2024PsychoGAT: A Novel Psychological Measurement Paradigm through Interactive Fiction Games with LLM Agentscited by 28
  6. 2024TeachTune: Reviewing Pedagogical Agents Against Diverse Student Profiles with Simulated Studentscited by 25
  7. 2025Prototypical Human-AI Collaboration Behaviors from LLM-Assisted Writing in the Wildcited by 25
  8. 2024Can LLMs Reliably Simulate Human Learner Actions? A Simulation Authoring Framework for Open-Ended Learning Environmentscited by 17
  9. 2024LLM-based Cognitive Models of Students with Misconceptionscited by 16
  10. 2025Classroom Simulacra: Building Contextual Student Generative Agents in Online Education for Learning Behavioral Simulationcited by 15
  11. 2025Do LLMs Give Psychometrically Plausible Responses in Educational Assessments?cited by 14
  12. 2025TutorGym: A Testbed for Evaluating AI Agents as Tutors and Studentscited by 10
  13. 2025Which Type of Students can LLMs Act? Investigating Authentic Simulation with Graph-based Human-AI Collaborative Systemcited by 9
  14. 202461A Bot Report: AI Assistants in CS1 Save Students Homework Time and Reduce Demands on Staff. (Now What?)cited by 7
  15. 2024LLM-itation is the Sincerest Form of Data: Generating Synthetic Buggy Code Submissions for Computing Educationcited by 7
  16. 2024Synthetic Students: A Comparative Study of Bug Distribution Between Large Language Models and Computing Studentscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026INSIDE the Student's Mind: Jointly Modeling Latent Reasoning and Action in LLM Student Simulatorscited by 1
  2. 2026Recon: Reconstruction-Guided Reasoning Synthesis for User Modelingcited by 0