# From Solver to Tutor: Evaluating the Pedagogical Intelligence of LLMs with KMP-Bench

arXiv: https://arxiv.org/abs/2603.02775 · HTML: https://mlsanity.com/paper/2603.02775/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 29 references that are in the map.
- First posted on arXiv 3 Mar 2026.

## Builds on (most cited first, 16 of 16)

- [Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement](https://mlsanity.com/paper/2409.12122/) (2024)
- [WildChat: 1M ChatGPT Interaction Logs in the Wild](https://mlsanity.com/paper/2405.01470/) (2024)
- [ProcessBench: Identifying Process Errors in Mathematical Reasoning](https://mlsanity.com/paper/2412.06559/) (2024)
- [MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning](https://mlsanity.com/paper/2310.03731/) (2023)
- [MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems](https://mlsanity.com/paper/2305.14536/) (2023)
- [Unifying AI Tutor Evaluation: An Evaluation Taxonomy for Pedagogical Ability Assessment of LLM-Powered AI Tutors](https://mlsanity.com/paper/2412.09416/) (2024)
- [MR-GSM8K: A Meta-Reasoning Benchmark for Large Language Model Evaluation](https://mlsanity.com/paper/2312.17080/) (2023)
- [Can Large Language Models be Good Emotional Supporter? Mitigating Preference Bias on Emotional Support Conversation](https://mlsanity.com/paper/2402.13211/) (2024)
- [MathTutorBench: A Benchmark for Measuring Open-ended Pedagogical Capabilities of LLM Tutors](https://mlsanity.com/paper/2502.18940/) (2025)
- [AutoTutor meets Large Language Models: A Language Model Tutor with Rich Pedagogy and Guardrails](https://mlsanity.com/paper/2402.09216/) (2024)
- [MathChat: Benchmarking Mathematical Reasoning and Instruction Following in Multi-Turn Interactions](https://mlsanity.com/paper/2405.19444/) (2024)
- [Boosting Large Language Models with Socratic Method for Conversational Mathematics Teaching](https://mlsanity.com/paper/2407.17349/) (2024)
- [SuperCLUE-Math6: Graded Multi-Step Math Reasoning Benchmark for LLMs in Chinese](https://mlsanity.com/paper/2401.11819/) (2024)
- [MATHWELL: Generating Educational Math Word Problems Using Teacher Annotations](https://mlsanity.com/paper/2402.15861/) (2024)
- [Mathfish: Evaluating Language Model Math Reasoning via Grounding in Educational Curricula](https://mlsanity.com/paper/2408.04226/) (2024)
- [Exploring the Effects of AI-assisted Emotional Support Processes in Online Mental Health Community](https://mlsanity.com/paper/2202.10065/) (2022)

## Built on by (most cited first, 2 of 2)

- [OmniEdu: Open Foundation Models for Learning and Teaching](https://mlsanity.com/paper/2609.23088/) (2026)
- [DICE: Decoupling Capability from Intervention Necessity in LLM Tutoring](https://mlsanity.com/paper/2610.04825/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) show significant potential in AI mathematical tutoring, yet current evaluations often rely on simplistic metrics or narrow pedagogical scenarios, failing to assess comprehensive, multi-turn teaching effectiveness. In this paper, we introduce KMP-Bench, a comprehensive K-8 Mathematical Pedagogical Benchmark designed to assess LLMs from two complementary perspectives. The first module, KMP-Dialogue, evaluates holistic pedagogical capabilities against six core principles (e.g., Challenge, Explanation, Feedback), leveraging a novel multi-turn dialogue dataset constructed by weaving together diverse pedagogical components. The second module, KMP-Skills, provides a granular assessment of foundational tutoring abilities, including multi-turn problem-solving, error detection and correction, and problem generation. Our evaluations on KMP-Bench reveal a key disparity: while leading LLMs excel at tasks with verifiable solutions, they struggle with the nuanced application of pedagogical principles. Additionally, we present KMP-Pile, a large-scale (150K) dialogue dataset. Models fine-tuned on KMP-Pile show substantial improvement on KMP-Bench, underscoring the value of pedagogically-rich training data for developing more effective AI math tutors.