# TeachTune: Reviewing Pedagogical Agents Against Diverse Student Profiles with Simulated Students

arXiv: https://arxiv.org/abs/2410.04078 · HTML: https://mlsanity.com/paper/2410.04078/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Placed in Educational agents and math problem solving, one of 16 directions in LLM agents and tool use.
- Builds on 44 of its 141 references that are in the map.
- First posted on arXiv 5 Oct 2024.

## Placed in

- LLM agents and tool use / [Educational agents and math problem solving](https://mlsanity.com/agents/educational-agents-and-math-problem-solving/)

## Builds on (most cited first, 30 of 44)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Character-LLM: A Trainable Agent for Role-Playing](https://mlsanity.com/paper/2310.10158/) (2023)
- [AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts](https://mlsanity.com/paper/2110.01691/) (2021)
- [PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits](https://mlsanity.com/paper/2305.02547/) (2023)
- [The What-If Tool: Interactive Probing of Machine Learning Models](https://mlsanity.com/paper/1907.04135/) (2019)
- [EconAgent: Large Language Model-Empowered Agents for Simulating Macroeconomic Activities](https://mlsanity.com/paper/2310.10436/) (2023)
- [Simulating Opinion Dynamics with Networks of LLM-based Agents](https://mlsanity.com/paper/2311.09618/) (2023)
- [Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordia](https://mlsanity.com/paper/2312.03664/) (2023)
- [ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testing](https://mlsanity.com/paper/2309.09128/) (2023)
- [EvalLM: Interactive Evaluation of Large Language Model Prompts on User-Defined Criteria](https://mlsanity.com/paper/2309.13633/) (2023)
- [A Complete Survey on LLM-based AI Chatbots](https://mlsanity.com/paper/2406.16937/) (2024)
- [CogBench: a large language model walks into a psychology lab](https://mlsanity.com/paper/2402.18225/) (2024)
- [Generative Students: Using LLM-Simulated Student Profiles to Support Question Item Evaluation](https://mlsanity.com/paper/2405.11591/) (2024)
- [How should my chatbot interact? A survey on human-chatbot interaction design](https://mlsanity.com/paper/1904.02743/) (2019)
- [Rehearsal: Simulating Conflict to Teach Conflict Resolution](https://mlsanity.com/paper/2309.12309/) (2023)
- [Towards Responsible Development of Generative AI for Education: An Evaluation-Driven Approach](https://mlsanity.com/paper/2407.12687/) (2024)
- [CLASS: A Design Framework for building Intelligent Tutoring Systems based on Learning Science principles](https://mlsanity.com/paper/2305.13272/) (2023)
- [EduAgent: Generative Student Agents in Learning](https://mlsanity.com/paper/2404.07963/) (2024)
- [Teach AI How to Code: Using Large Language Models as Teachable Agents for Programming Education](https://mlsanity.com/paper/2309.14534/) (2023)
- [Zeno: An Interactive Framework for Behavioral Evaluation of Machine Learning](https://mlsanity.com/paper/2302.04732/) (2023)
- [ConstitutionMaker: Interactively Critiquing Large Language Models by Converting Feedback into Principles](https://mlsanity.com/paper/2310.15428/) (2023)
- [Calendar.help: Designing a Workflow-Based Scheduling Agent with Humans in the Loop](https://mlsanity.com/paper/1703.08428/) (2017)
- [PATIENT-Psi: Using Large Language Models to Simulate Patients for Training Mental Health Professionals](https://mlsanity.com/paper/2405.19660/) (2024)
- [ScatterShot: Interactive In-context Example Curation for Text Transformation](https://mlsanity.com/paper/2302.07346/) (2023)
- [PeerGPT: Probing the Roles of LLM-based Peer Agents as Team Moderators and Participants in Children's Collaborative Learning](https://mlsanity.com/paper/2403.14227/) (2024)

## Built on by (most cited first, 25 of 25)

- [LLM Agents for Education: Advances and Applications](https://mlsanity.com/paper/2503.11733/) (2025)
- [EducationQ: Evaluating LLMs' Teaching Capabilities Through Multi-Agent Dialogue Framework](https://mlsanity.com/paper/2504.14928/) (2025)
- [LLM-Powered AI Agent Systems and Their Applications in Industry](https://mlsanity.com/paper/2505.16120/) (2025)
- [A Survey on LLM-based Conversational User Simulation](https://mlsanity.com/paper/2604.24977/) (2026)
- [Toward Personalized LLM-Powered Agents: Foundations, Evaluation, and Future Directions](https://mlsanity.com/paper/2602.22680/) (2026)
- [Evolution of AI in Education: Agentic Workflows](https://mlsanity.com/paper/2504.20082/) (2025)
- [Which Type of Students can LLMs Act? Investigating Authentic Simulation with Graph-based Human-AI Collaborative System](https://mlsanity.com/paper/2502.11678/) (2025)
- [Towards Valid Student Simulation with Large Language Models](https://mlsanity.com/paper/2601.05473/) (2026)
- [Simulating Students with Large Language Models: A Review of Architecture, Mechanisms, and Role Modelling in Education with Generative AI](https://mlsanity.com/paper/2511.06078/) (2025)
- [One Model, Many Morals: Uncovering Cross-Linguistic Misalignments in Computational Moral Reasoning](https://mlsanity.com/paper/2509.21443/) (2025)
- [PATS: Personality-Aware Teaching Strategies with Large Language Model Tutors](https://mlsanity.com/paper/2601.08402/) (2026)
- [ParaStudent: Closing the Sim2Real Gap in User Simulators for AI Tutor Evaluation](https://mlsanity.com/paper/2507.12674/) (2025)
- [What Makes LLM Agent Simulations Useful for Policy Practice? An Iterative Design Study in Emergency Preparedness](https://mlsanity.com/paper/2509.21868/) (2025)
- [Free Lunch for User Experience: Crowdsourcing Agents for Scalable User Studies](https://mlsanity.com/paper/2505.22981/) (2025)
- [DrawSim-PD: Simulating Student Science Drawings to Support NGSS-Aligned Teacher Diagnostic Reasoning](https://mlsanity.com/paper/2602.01578/) (2026)
- [A Scoping Review of Large Language Model-Based Pedagogical Agents](https://mlsanity.com/paper/2604.12253/) (2026)
- [Student Development Agent: Risk-free Simulation for Evaluating AIED Innovations](https://mlsanity.com/paper/2510.09183/) (2025)
- [Towards Synergistic Teacher-AI Interactions with Generative Artificial Intelligence](https://mlsanity.com/paper/2511.19580/) (2025)
- [The RIGID Framework: Research-Integrated, Generative AI-Mediated Instructional Design](https://mlsanity.com/paper/2603.12781/) (2026)
- [Voting Protocols as Coordination Mechanisms for Role-Constrained Multi-Agent Tutoring Systems](https://mlsanity.com/paper/2606.08030/) (2026)
- [Lect=uraAgents: A Multi-Agent Framework for Adaptive Personalized AI-Assisted Learning and Embodied Teaching](https://mlsanity.com/paper/2606.16428/) (2026)
- [AI Assistants Overassist](https://mlsanity.com/paper/2607.21306/) (2026)
- [Eco3S: Complex Socio-Economic System Simulation via Agent-Based Models](https://mlsanity.com/paper/2607.26588/) (2026)
- [ELBench: A Multi-Dimensional Benchmark for Education-Facing Large Language Models](https://mlsanity.com/paper/2608.09548/) (2026)
- [From Learner Behavior to Reusable Skills for Effective and Efficient Learner Simulation](https://mlsanity.com/paper/2609.37157/) (2026)

## Abstract (arXiv)

Large language models (LLMs) can empower teachers to build pedagogical conversational agents (PCAs) customized for their students. As students have different prior knowledge and motivation levels, teachers must review the adaptivity of their PCAs to diverse students. Existing chatbot reviewing methods (e.g., direct chat and benchmarks) are either manually intensive for multiple iterations or limited to testing only single-turn interactions. We present TeachTune, where teachers can create simulated students and review PCAs by observing automated chats between PCAs and simulated students. Our technical pipeline instructs an LLM-based student to simulate prescribed knowledge levels and traits, helping teachers explore diverse conversation patterns. Our pipeline could produce simulated students whose behaviors correlate highly to their input knowledge and motivation levels within 5% and 10% accuracy gaps. Thirty science teachers designed PCAs in a between-subjects study, and using TeachTune resulted in a lower task load and higher student profile coverage over a baseline.