mlsanityData to 7 Oct 2026

TeachTune: Reviewing Pedagogical Agents Against Diverse Student Profiles with Simulated Students

arXiv 2410.04078 ↗First posted 5 Oct 2024cs.HCcited by 25

On the map: LLM agents and tool use → Educational agents and math problem solving

Abstract (arXiv)

Large language models (LLMs) can empower teachers to build pedagogical conversational agents (PCAs) customized for their students. As students have different prior knowledge and motivation levels, teachers must review the adaptivity of their PCAs to diverse students. Existing chatbot reviewing methods (e.g., direct chat and benchmarks) are either manually intensive for multiple iterations or limited to testing only single-turn interactions. We present TeachTune, where teachers can create simulated students and review PCAs by observing automated chats between PCAs and simulated students. Our technical pipeline instructs an LLM-based student to simulate prescribed knowledge levels and traits, helping teachers explore diverse conversation patterns. Our pipeline could produce simulated students whose behaviors correlate highly to their input knowledge and motivation levels within 5% and 10% accuracy gaps. Thirty science teachers designed PCAs in a between-subjects study, and using TeachTune resulted in a lower task load and higher student profile coverage over a baseline.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Placed in Educational agents and math problem solving, one of 16 directions in LLM agents and tool use. Builds on 44 of its 141 references that are in the map. First posted on arXiv 5 Oct 2024.

Builds on

44 of 141 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024EduAgent: Generative Student Agents in Learning23 Mar
  2. 2024PeerGPT: Probing the Roles of LLM-based Peer Agents as Team Moderators and Participants in Children's Collaborative Learning21 Mar
  3. 2023Teach AI How to Code: Using Large Language Models as Teachable Agents for Programming Education25 Sep

Other references · 41

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  4. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  5. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  6. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  7. 2023Character-LLM: A Trainable Agent for Role-Playingcited by 335
  8. 2021AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Promptscited by 265
  9. 2023PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traitscited by 229
  10. 2019The What-If Tool: Interactive Probing of Machine Learning Modelscited by 207
  11. 2023EconAgent: Large Language Model-Empowered Agents for Simulating Macroeconomic Activitiescited by 154
  12. 2023Simulating Opinion Dynamics with Networks of LLM-based Agentscited by 153
29 more
  1. 2023Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordiacited by 119
  2. 2023ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testingcited by 88
  3. 2023EvalLM: Interactive Evaluation of Large Language Model Prompts on User-Defined Criteriacited by 73
  4. 2024A Complete Survey on LLM-based AI Chatbotscited by 71
  5. 2024CogBench: a large language model walks into a psychology labcited by 60
  6. 2019How should my chatbot interact? A survey on human-chatbot interaction designcited by 56
  7. 2024Generative Students: Using LLM-Simulated Student Profiles to Support Question Item Evaluationcited by 56
  8. 2023Rehearsal: Simulating Conflict to Teach Conflict Resolutioncited by 54
  9. 2024Towards Responsible Development of Generative AI for Education: An Evaluation-Driven Approachcited by 53
  10. 2023CLASS: A Design Framework for building Intelligent Tutoring Systems based on Learning Science principlescited by 46
  11. 2023Zeno: An Interactive Framework for Behavioral Evaluation of Machine Learningcited by 33
  12. 2023ConstitutionMaker: Interactively Critiquing Large Language Models by Converting Feedback into Principlescited by 33
  13. 2017Calendar.help: Designing a Workflow-Based Scheduling Agent with Humans in the Loopcited by 26
  14. 2023ScatterShot: Interactive In-context Example Curation for Text Transformationcited by 20
  15. 2024PATIENT-Psi: Using Large Language Models to Simulate Patients for Training Mental Health Professionalscited by 19
  16. 2023RECIPE: How to Integrate ChatGPT into EFL Writing Educationcited by 17
  17. 2023Unraveling Downstream Gender Bias from Large Language Models: A Study on AI Educational Writing Assistancecited by 15
  18. 2024Evolving Agents: Interactive Simulation of Dynamic and Diverse Human Personalitiescited by 14
  19. 2023ChatGPT in education: A discourse analysis of worries and concerns on social mediacited by 12
  20. 2024Evaluating and Optimizing Educational Content with Large Language Model Judgmentscited by 11
  21. 2024A Piece of Theatre: Investigating How Teachers Design LLM Chatbots to Assist Adolescent Cyberbullying Educationcited by 10
  22. 2024Generating Educational Materials with Different Levels of Readability using LLMscited by 9
  23. 2024Using LLMs to Model the Beliefs and Preferences of Targeted Populationscited by 6
  24. 2020Pedagogical Agents for Fostering Question-Asking Skills in Childrencited by 5
  25. 2024Towards Educator-Driven Tutor Authoring: Generative AI Approaches for Creating Intelligent Tutor Interfacescited by 5
  26. 2024Towards Generalizable Agents in Text-Based Educational Environments: A Study of Integrating RL with LLMscited by 4
  27. 2024On LLM Wizards: Identifying Large Language Models' Behaviors for Wizard of Oz Experimentscited by 4
  28. 2019Predicting Student Performance in an Educational Game Using a Hidden Markov Modelcited by 3
  29. 2024Subgroup Discovery in MOOCs: A Big Data Application for Describing Different Types of Learnerscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 12; 2026: 13.

25 papers.

Sort:

By area:

  1. 2025LLM Agents for Education: Advances and Applicationscited by 59
  2. 2025EducationQ: Evaluating LLMs' Teaching Capabilities Through Multi-Agent Dialogue Frameworkcited by 23
  3. 2025LLM-Powered AI Agent Systems and Their Applications in Industrycited by 22
  4. 2026A Survey on LLM-based Conversational User Simulationcited by 14
  5. 2025Evolution of AI in Education: Agentic Workflowscited by 13
  6. 2026Toward Personalized LLM-Powered Agents: Foundations, Evaluation, and Future Directionscited by 10
  7. 2025Which Type of Students can LLMs Act? Investigating Authentic Simulation with Graph-based Human-AI Collaborative Systemcited by 9
  8. 2026Towards Valid Student Simulation with Large Language Modelscited by 7
  9. 2025Simulating Students with Large Language Models: A Review of Architecture, Mechanisms, and Role Modelling in Education with Generative AIcited by 5
  10. 2025One Model, Many Morals: Uncovering Cross-Linguistic Misalignments in Computational Moral Reasoningcited by 4
  11. 2026PATS: Personality-Aware Teaching Strategies with Large Language Model Tutorscited by 4
  12. 2025ParaStudent: Closing the Sim2Real Gap in User Simulators for AI Tutor Evaluationcited by 2
13 more
  1. 2025What Makes LLM Agent Simulations Useful for Policy Practice? An Iterative Design Study in Emergency Preparednesscited by 2
  2. 2025Free Lunch for User Experience: Crowdsourcing Agents for Scalable User Studiescited by 1
  3. 2026DrawSim-PD: Simulating Student Science Drawings to Support NGSS-Aligned Teacher Diagnostic Reasoningcited by 1
  4. 2026A Scoping Review of Large Language Model-Based Pedagogical Agentscited by 1
  5. 2025Student Development Agent: Risk-free Simulation for Evaluating AIED Innovationscited by 0
  6. 2025Towards Synergistic Teacher-AI Interactions with Generative Artificial Intelligencecited by 0
  7. 2026The RIGID Framework: Research-Integrated, Generative AI-Mediated Instructional Designcited by 0
  8. 2026Voting Protocols as Coordination Mechanisms for Role-Constrained Multi-Agent Tutoring Systemscited by 0
  9. 2026Lect=uraAgents: A Multi-Agent Framework for Adaptive Personalized AI-Assisted Learning and Embodied Teachingcited by 0
  10. 2026AI Assistants Overassistcited by 0
  11. 2026Eco3S: Complex Socio-Economic System Simulation via Agent-Based Modelscited by 0
  12. 2026ELBench: A Multi-Dimensional Benchmark for Education-Facing Large Language Modelscited by 0
  13. 2026From Learner Behavior to Reusable Skills for Effective and Efficient Learner Simulationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Conversational Education at Scale: A Multi-LLM Agent Workflow for Procedural Learning and Pedagogic Quality Assessment
  2. 2024Ruffle&Riley: Insights from Designing and Evaluating a Large Language Model-Based Conversational Tutoring System
  3. 2025Embracing Imperfection: Simulating Students with Diverse Cognitive Levels Using LLM-based Agents
  4. 2025Dialogic Pedagogy for Large Language Models: Aligning Conversational AI with Proven Theories of Learning
  5. 2025MASTER: Enhancing Large Language Model via Multi-Agent Simulated Teaching
  6. 2026INSIDE the Student's Mind: Jointly Modeling Latent Reasoning and Action in LLM Student Simulators
  7. 2023The Use of Multiple Conversational Agent Interlocutors in Learning
  8. 2026Identifying High-Confidence Social Biases in LLMs for Trustworthy Conversational Tutoring Agents