Large Language Models Approach Expert Pedagogical Quality in Math Tutoring but Differ in Instructional and Linguistic Profiles
Not on the map. Why?
Abstract (arXiv)
Recent work has explored the use of large language models (LLMs) to generate tutoring responses in mathematics, yet it remains unclear how closely their instructional behavior aligns with expert human practice. We analyze a dataset of math remediation dialogues in which expert tutors, novice tutors, and seven LLMs of varying sizes, comprising both open-weight and commercial models, respond to the same student errors. We examine instructional strategies and linguistic characteristics of tutoring responses, including uptake (restating and revoicing), pressing for accuracy and reasoning, lexical diversity, readability, politeness, and agency. We find that expert tutors produce higher-quality responses than novices, and that larger LLMs generally receive higher pedagogical quality ratings than smaller models, approaching expert performance on average. However, LLMs exhibit systematic differences in their instructional profiles: they underuse discursive strategies characteristic of expert tutors while generating longer, more lexically diverse, and more polite responses. Regression analyses show that pressing for accuracy and reasoning, restating and revoicing, and lexical diversity, are positively associated with perceived pedagogical quality, whereas higher levels of agentic and polite language are negatively associated. These findings highlight the importance of analyzing instructional strategies and linguistic characteristics when evaluating tutoring responses across human tutors and intelligent tutoring systems.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 41 references that are in the map. First posted on arXiv 23 Dec 2025.
Builds on
18 of 41 references.
References · 18
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
- 2023MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problemscited by 128
- 2024Unifying AI Tutor Evaluation: An Evaluation Taxonomy for Pedagogical Ability Assessment of LLM-Powered AI Tutorscited by 60
- 2023Bridging the Novice-Expert Gap via Models of Decision-Making: A Case Study on Remediating Math Mistakescited by 51
- 2024Evaluating Mathematical Reasoning of Large Language Models: A Focus on Error Identification and Correctioncited by 45
- 2024Detection and Measurement of Syntactic Templates in Generated Textcited by 44
- 2022The TalkMoves Dataset: K-12 Mathematics Lesson Transcripts Annotated for Teacher and Student Discursive Movescited by 38
- 2024AutoTutor meets Large Language Models: A Language Model Tutor with Rich Pedagogy and Guardrailscited by 30
- 2022TyDiP: A Dataset for Politeness Classification in Nine Typologically Diverse Languagescited by 21
- 2025Training Turn-by-Turn Verifiers for Dialogue Tutoring Agents: The Curious Case of LLMs as Your Coding Tutorscited by 17
- 2025Findings of the BEA 2025 Shared Task on Pedagogical Ability Assessment of AI-powered Tutorscited by 16
6 more
- 2023Novice Learner and Expert Tutor: Evaluating Math Reasoning Abilities of Large Language Models with Misconceptionscited by 15
- 2025QUDsim: Quantifying Discourse Similarities in LLM-Generated Textcited by 13
- 2024Using Large Language Models to Assess Tutors' Performance in Reacting to Students Making Math Errorscited by 9
- 2025Linguistic and Embedding-Based Profiling of Texts generated by Humans and Large Language Modelscited by 6
- 2024Computational analysis of US Congressional speeches reveals a shift from evidence to intuitioncited by 4
- 2023Collective moderation of hate, toxicity, and extremity in online discussionscited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Simulating LLM-to-LLM Tutoring for Multilingual Math Feedback
- 2025MathTutorBench: A Benchmark for Measuring Open-ended Pedagogical Capabilities of LLM Tutors
- 2026Is Mathematical Problem-Solving Expertise in Large Language Models Associated with Assessment Performance?
- 2026Large Language Models as Students Who Think Aloud: Overly Coherent, Verbose, and Confident
- 2023Three Questions Concerning the Use of Large Language Models to Facilitate Mathematics Learning
- 2025MathRobust-LV: Evaluation of Large Language Models' Robustness to Linguistic Variations in Mathematical Reasoning
- 2024Stepwise Verification and Remediation of Student Reasoning Errors with Large Language Model Tutors
- 2026Towards Pedagogically Aligned LLM Tutors for Math Mistake Remediation