mlsanityData to 14 Oct 2026

Knowledge Distillation for Automated AI Tutor Evaluation

arXiv 2607.10647 ↗First posted 12 Jul 2026cs.CLcited by 0

On the map: Knowledge distillation → Knowledge distillation for LLMs

Abstract (arXiv)

The rapid integration of Large Language Models (LLMs) into K-12 and higher education has outpaced the development of reliable methods for evaluating their pedagogical quality. As the research community starts to explore the space of automating evaluation of AI tutors, we introduce FATE (FLC AI Tutor Evaluator), a specialized 8B-parameter language model designed to evaluate AI tutors. Aligned with the four core evaluation tracks from the BEA 2025 Shared Task, our model assesses pedagogical ability across Mistake Identification, Mistake Location, Guidance, and Actionability. Because pedagogical evaluation is a specialized task with limited labeled data, we leverage knowledge distillation from a frontier LLM to generate additional supervision, yielding absolute performance gains up to 22.63 percentage points. Finally, we demonstrate FATE's utility as an automated evaluator by benchmarking instructional responses generated by popular commercial models, including ChatGPT, Claude, Gemini, and DeepSeek. On average, we have found that Gemini 2.5 Flash perfomed best (82.88%), then ChatGPT 5.5 Instant (80.75%), followed by DeepSeek V4 Flash (80.13%) and Claude Sonnet 4.6 (74.00%).

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Knowledge distillation for LLMs, one of 12 directions in Knowledge distillation. Builds on 11 of its 16 references that are in the map. First posted on arXiv 12 Jul 2026.

Builds on

11 of 16 references.

References · 11

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  3. 2024Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Modelscited by 446
  4. 2021Does Knowledge Distillation Really Work?cited by 190
  5. 2022The AI Teacher Test: Measuring the Pedagogical Ability of Blender and GPT-3 in Educational Dialoguescited by 79
  6. 2024Unifying AI Tutor Evaluation: An Evaluation Taxonomy for Pedagogical Ability Assessment of LLM-Powered AI Tutorscited by 61
  7. 2023The BEA 2023 Shared Task on Generating AI Teacher Responses in Educational Dialoguescited by 27
  8. 2025Findings of the BEA 2025 Shared Task on Pedagogical Ability Assessment of AI-powered Tutorscited by 18
  9. 2025TutorBench: A Benchmark To Assess Tutoring Capabilities Of Large Language Modelscited by 11
  10. 2025TutorGym: A Testbed for Evaluating AI Agents as Tutors and Studentscited by 10
  11. 2026A Dialogue-Based Framework for Correcting Multimodal Errors in AI-Assisted STEM Educationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025MathTutorBench: A Benchmark for Measuring Open-ended Pedagogical Capabilities of LLM Tutors
  2. 2026GRADE: Generalizable Reasoning-Aware Dialogue Evaluation for AI Tutors
  3. 2026Towards Reward Modeling for AI Tutors in Math Mistake Remediation
  4. 2025Dean of LLM Tutors: A Framework for Automated Quality Review of AI-generated Feedback
  5. 2025NeuralNexus at BEA 2025 Shared Task: Retrieval-Augmented Prompting for Mistake Identification in AI Tutors
  6. 2024Evaluating the Impact of Advanced LLM Techniques on AI-Lecture Tutors for a Robotics Course
  7. 2023Knowledge Distillation of LLM for Automatic Scoring of Science Education Assessments
  8. 2025Honey, I Shrunk the Language Model: Impact of Knowledge Distillation Methods on Performance and Explainability