# Training LLM-based Tutors to Improve Student Learning Outcomes in Dialogues

arXiv: https://arxiv.org/abs/2503.06424 · HTML: https://mlsanity.com/paper/2503.06424/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 55 references that are in the map.
- First posted on arXiv 9 Mar 2025.

## Builds on (most cited first, 30 of 32)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Deep Knowledge Tracing](https://mlsanity.com/paper/1506.05908/) (2015)
- [Context-Aware Attentive Knowledge Tracing](https://mlsanity.com/paper/2007.12324/) (2020)
- [MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems](https://mlsanity.com/paper/2305.14536/) (2023)
- [Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinement](https://mlsanity.com/paper/2402.11436/) (2024)
- [Survey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Application](https://mlsanity.com/paper/2407.01885/) (2024)
- [Bridging the Novice-Expert Gap via Models of Decision-Making: A Case Study on Remediating Math Mistakes](https://mlsanity.com/paper/2310.10648/) (2023)
- [Tutor CoPilot: A Human-AI Approach for Scaling Real-Time Expertise](https://mlsanity.com/paper/2410.03017/) (2024)
- [CLASS: A Design Framework for building Intelligent Tutoring Systems based on Learning Science principles](https://mlsanity.com/paper/2305.13272/) (2023)
- [Exploring Knowledge Tracing in Tutor-Student Dialogues using LLMs](https://mlsanity.com/paper/2409.16490/) (2024)
- [Measuring Conversational Uptake: A Case Study on Student-Teacher Interactions](https://mlsanity.com/paper/2106.03873/) (2021)
- [Enhancing LLM-Based Feedback: Insights from Intelligent Tutoring Systems and the Learning Sciences](https://mlsanity.com/paper/2405.04645/) (2024)
- [Empowering Personalized Learning through a Conversation-based Tutoring System with Student Modeling](https://mlsanity.com/paper/2403.14071/) (2024)
- [AutoTutor meets Large Language Models: A Language Model Tutor with Rich Pedagogy and Guardrails](https://mlsanity.com/paper/2402.09216/) (2024)
- [LLM-as-a-tutor in EFL Writing Education: Focusing on Evaluation of Student-LLM Interaction](https://mlsanity.com/paper/2310.05191/) (2023)
- [Ruffle&Riley: Insights from Designing and Evaluating a Large Language Model-Based Conversational Tutoring System](https://mlsanity.com/paper/2404.17460/) (2024)
- [Improving the Validity of Automatically Generated Feedback via Reinforcement Learning](https://mlsanity.com/paper/2403.01304/) (2024)
- [Adaptive and Personalized Exercise Generation for Online Language Learning](https://mlsanity.com/paper/2306.02457/) (2023)
- [Pedagogical Alignment of Large Language Models](https://mlsanity.com/paper/2402.05000/) (2024)
- [Improving Socratic Question Generation using Data Augmentation and Preference Optimization](https://mlsanity.com/paper/2403.00199/) (2024)
- [Novice Learner and Expert Tutor: Evaluating Math Reasoning Abilities of Large Language Models with Misconceptions](https://mlsanity.com/paper/2310.02439/) (2023)
- [CREF: An LLM-based Conversational Software Repair Framework for Programming Tutors](https://mlsanity.com/paper/2406.13972/) (2024)
- [Margin Matching Preference Optimization: Enhanced Model Alignment with Granular Feedback](https://mlsanity.com/paper/2410.03145/) (2024)
- [Evaluating and Optimizing Educational Content with Large Language Model Judgments](https://mlsanity.com/paper/2403.02795/) (2024)
- [Challenges and Opportunities of Moderating Usage of Large Language Models in Education](https://mlsanity.com/paper/2312.14969/) (2023)
- [Using Large Language Models to Assess Tutors' Performance in Reacting to Students Making Math Errors](https://mlsanity.com/paper/2401.03238/) (2024)
- [Automatic Pair Construction for Contrastive Post-training](https://mlsanity.com/paper/2310.02263/) (2023)

## Built on by (most cited first, 30 of 43)

- [SMART: Simulated Students Aligned with Item Response Theory for Question Difficulty Prediction](https://mlsanity.com/paper/2507.05129/) (2025)
- [Simulated Students in Tutoring Dialogues: Substance or Illusion?](https://mlsanity.com/paper/2601.04025/) (2026)
- [From Problem-Solving to Teaching Problem-Solving: Aligning LLMs with Pedagogy using Reinforcement Learning](https://mlsanity.com/paper/2505.15607/) (2025)
- [TutorBench: A Benchmark To Assess Tutoring Capabilities Of Large Language Models](https://mlsanity.com/paper/2510.02663/) (2025)
- [Offline RL by Reward-Weighted Fine-Tuning for Conversation Optimization](https://mlsanity.com/paper/2506.06964/) (2025)
- [SoM-1K: A Thousand-Problem Benchmark Dataset for Strength of Materials](https://mlsanity.com/paper/2509.21079/) (2025)
- [LookAlike: Consistent Distractor Generation in Math MCQs](https://mlsanity.com/paper/2505.01903/) (2025)
- [Evaluating Answer Leakage Robustness of LLM Tutors against Adversarial Student Attacks](https://mlsanity.com/paper/2604.18660/) (2026)
- [Discerning minds or generic tutors? Evaluating instructional guidance capabilities in Socratic LLMs](https://mlsanity.com/paper/2508.06583/) (2025)
- [Reinforcement Learning in the Real World: A Survey of Statistical Challenges and Future Directions](https://mlsanity.com/paper/2601.15353/) (2026)
- [Heterogeneous Debate Engine: Identity-Grounded Cognitive Architecture for Resilient LLM-Based Ethical Tutoring](https://mlsanity.com/paper/2603.27404/) (2026)
- [Exploring LLMs for Predicting Tutor Strategy and Student Outcomes in Dialogues](https://mlsanity.com/paper/2507.06910/) (2025)
- [VEHME: A Vision-Language Model For Evaluating Handwritten Mathematics Expressions](https://mlsanity.com/paper/2510.22798/) (2025)
- [Letting Tutor Personas Speak Up for LLMs: Learning Steering Vectors from Dialogue via Preference Optimization](https://mlsanity.com/paper/2602.07639/) (2026)
- [From Untamed Black Box to Interpretable Pedagogical Orchestration: The Ensemble of Specialized LLMs Architecture for Adaptive Tutoring](https://mlsanity.com/paper/2603.23990/) (2026)
- [A Meta-Analysis of LLM Effects on Students across Qualification, Socialisation, and Subjectification](https://mlsanity.com/paper/2509.22725/) (2025)
- [Enhancing Diversity of LLM-Generated Educational Tasks](https://mlsanity.com/paper/2512.23601/) (2025)
- [AgentTutor: Empowering Personalized Learning with Multi-Turn Interactive Teaching in Intelligent Education Systems](https://mlsanity.com/paper/2601.04219/) (2025)
- [LLMs as Science Journalists: Supporting Early-stage Researchers in Communicating Their Science to the Public](https://mlsanity.com/paper/2601.05821/) (2026)
- [Interpretable Difficulty-Aware Knowledge Tracing in Tutor-Student Dialogues](https://mlsanity.com/paper/2605.01097/) (2026)
- [Cultivating Helpful, Personalized, and Creative AI Tutors: A Framework for Pedagogical Alignment using Reinforcement Learning](https://mlsanity.com/paper/2507.20335/) (2025)
- [Teaching Language Models How to Code Like Learners: Conversational Serialization for Student Simulation](https://mlsanity.com/paper/2604.10720/) (2026)
- [Toward a Benchmark for Controllable Simulation of Imperfect Students with Large Language Models](https://mlsanity.com/paper/2605.25601/) (2026)
- [PEARL: Training Socratic Tutors with Pedagogically Aligned Reinforcement Learning](https://mlsanity.com/paper/2605.29582/) (2026)
- [Hey Chat, Can You Teach Me? Structuring Socratic Dialogue for Human Learning in the Wild](https://mlsanity.com/paper/2606.11744/) (2026)
- [Multi-turn Conversational AI from Text to Multimodal Interaction: Data, Models, Evaluation, and Open Challenges](https://mlsanity.com/paper/2608.17605/) (2026)
- [StudentSim: Training LLM-based Student Simulators](https://mlsanity.com/paper/2609.01591/) (2026)
- [What Hard Tokens Reveal: Exploiting Low-confidence Tokens for Membership Inference Attacks against Large Language Models](https://mlsanity.com/paper/2601.20885/) (2026)
- [Hybrid LLM-Embedded Dialogue Agents for Learner Reflection: Designing Responsive and Theory-Driven Interactions](https://mlsanity.com/paper/2602.20486/) (2026)
- [Partial Policy Gradients for RL in LLMs](https://mlsanity.com/paper/2603.06138/) (2026)

## Abstract (arXiv)

Generative artificial intelligence (AI) has the potential to scale up personalized tutoring through large language models (LLMs). Recent AI tutors are adapted for the tutoring task by training or prompting LLMs to follow effective pedagogical principles, though they are not trained to maximize student learning throughout the course of a dialogue. Therefore, they may engage with students in a suboptimal way. We address this limitation by introducing an approach to train LLMs to generate tutor utterances that maximize the likelihood of student correctness, while still encouraging the model to follow good pedagogical practice. Specifically, we generate a set of candidate tutor utterances and score them using (1) an LLM-based student model to predict the chance of correct student responses and (2) a pedagogical rubric evaluated by GPT-4o. We then use the resulting data to train an open-source LLM, Llama 3.1 8B, using direct preference optimization. We show that tutor utterances generated by our model lead to significantly higher chances of correct student responses while maintaining the pedagogical quality of GPT-4o. We also conduct qualitative analyses and a human evaluation to demonstrate that our model generates high quality tutor utterances.