# Exploring LLMs for Predicting Tutor Strategy and Student Outcomes in Dialogues

arXiv: https://arxiv.org/abs/2507.06910 · HTML: https://mlsanity.com/paper/2507.06910/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 38 references that are in the map.
- First posted on arXiv 9 Jul 2025.

## Builds on (most cited first, 17 of 17)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems](https://mlsanity.com/paper/2305.14536/) (2023)
- [Bridging the Novice-Expert Gap via Models of Decision-Making: A Case Study on Remediating Math Mistakes](https://mlsanity.com/paper/2310.10648/) (2023)
- [Exploring Knowledge Tracing in Tutor-Student Dialogues using LLMs](https://mlsanity.com/paper/2409.16490/) (2024)
- [Measuring Conversational Uptake: A Case Study on Student-Teacher Interactions](https://mlsanity.com/paper/2106.03873/) (2021)
- [The NCTE Transcripts: A Dataset of Elementary Math Classroom Transcripts](https://mlsanity.com/paper/2211.11772/) (2022)
- [Training LLM-based Tutors to Improve Student Learning Outcomes in Dialogues](https://mlsanity.com/paper/2503.06424/) (2025)
- [The TalkMoves Dataset: K-12 Mathematics Lesson Transcripts Annotated for Teacher and Student Discursive Moves](https://mlsanity.com/paper/2204.09652/) (2022)
- [The BEA 2023 Shared Task on Generating AI Teacher Responses in Educational Dialogues](https://mlsanity.com/paper/2306.06941/) (2023)
- [Pedagogical Alignment of Large Language Models](https://mlsanity.com/paper/2402.05000/) (2024)
- [Edu-ConvoKit: An Open-Source Library for Education Conversation Data](https://mlsanity.com/paper/2402.05111/) (2024)
- [Effective and Efficient Conversation Retrieval for Dialogue State Tracking with Implicit Text Summaries](https://mlsanity.com/paper/2402.13043/) (2024)
- [Do LLMs Make Mistakes Like Students? Exploring Natural Alignment between Language Models and Human Error Patterns](https://mlsanity.com/paper/2502.15140/) (2025)
- [Knowledge Tagging System on Math Questions via LLMs with Flexible Demonstration Retriever](https://mlsanity.com/paper/2406.13885/) (2024)
- [What Would a Teacher Do? Predicting Future Talk Moves](https://mlsanity.com/paper/2106.05249/) (2021)
- [Aligning Tutor Discourse Supporting Rigorous Thinking with Tutee Content Mastery for Predicting Math Achievement](https://mlsanity.com/paper/2405.06218/) (2024)

## Built on by (most cited first, 4 of 4)

- [Simulated Students in Tutoring Dialogues: Substance or Illusion?](https://mlsanity.com/paper/2601.04025/) (2026)
- [MMTutorBench: The First Multimodal Benchmark for AI Math Tutoring](https://mlsanity.com/paper/2510.23477/) (2025)
- [Incentives Of EdTech: A Systematic Review Of EduNLP Research](https://mlsanity.com/paper/2606.13691/) (2026)
- [Do LLM Tutors Actually Teach? A Process-Centered Synthesis of the Tutoring Loop](https://mlsanity.com/paper/2610.15692/) (2026)

## Abstract (arXiv)

Tutoring dialogues have gained significant attention in recent years, given the prominence of online learning and the emerging tutoring abilities of artificial intelligence (AI) agents powered by large language models (LLMs). Recent studies have shown that the strategies used by tutors can have significant effects on student outcomes, necessitating methods to predict how tutors will behave and how their actions impact students. However, few works have studied predicting tutor strategy in dialogues. Therefore, in this work we investigate the ability of modern LLMs, particularly Llama 3 and GPT-4o, to predict both future tutor moves and student outcomes in dialogues, using two math tutoring dialogue datasets. We find that even state-of-the-art LLMs struggle to predict future tutor strategy while tutor strategy is highly indicative of student outcomes, outlining a need for more powerful methods to approach this task.