# EducationQ: Evaluating LLMs' Teaching Capabilities Through Multi-Agent Dialogue Framework

arXiv: https://arxiv.org/abs/2504.14928 · HTML: https://mlsanity.com/paper/2504.14928/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Placed in Educational agents and math problem solving, one of 16 directions in LLM agents and tool use.
- Builds on 18 of its 51 references that are in the map.
- First posted on arXiv 21 Apr 2025.

## Placed in

- LLM agents and tool use / [Educational agents and math problem solving](https://mlsanity.com/agents/educational-agents-and-math-problem-solving/)
- Knowledge distillation / [Knowledge distillation for LLMs](https://mlsanity.com/distill/knowledge-distillation-for-llms/)

## Builds on (most cited first, 18 of 18)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://mlsanity.com/paper/2406.01574/) (2024)
- [Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks](https://mlsanity.com/paper/2204.07705/) (2022)
- [Large Language Models are Better Reasoners with Self-Verification](https://mlsanity.com/paper/2212.09561/) (2022)
- [Benchmark Self-Evolving: A Multi-Agent Framework for Dynamic LLM Evaluation](https://mlsanity.com/paper/2402.11443/) (2024)
- [Generative Students: Using LLM-Simulated Student Profiles to Support Question Item Evaluation](https://mlsanity.com/paper/2405.11591/) (2024)
- [Automatic Generation of Socratic Subquestions for Teaching Math Word Problems](https://mlsanity.com/paper/2211.12835/) (2022)
- [A Chain-of-Thought Prompting Approach with LLMs for Evaluating Students' Formative Assessment Responses in Science](https://mlsanity.com/paper/2403.14565/) (2024)
- [MathChat: Benchmarking Mathematical Reasoning and Instruction Following in Multi-Turn Interactions](https://mlsanity.com/paper/2405.19444/) (2024)
- [TeachTune: Reviewing Pedagogical Agents Against Diverse Student Profiles with Simulated Students](https://mlsanity.com/paper/2410.04078/) (2024)
- [PeerGPT: Probing the Roles of LLM-based Peer Agents as Team Moderators and Participants in Children's Collaborative Learning](https://mlsanity.com/paper/2403.14227/) (2024)
- [Leveraging generative artificial intelligence to simulate student learning behavior](https://mlsanity.com/paper/2310.19206/) (2023)

## Built on by (most cited first, 23 of 23)

- [EduBench: A Comprehensive Benchmarking Dataset for Evaluating Large Language Models in Diverse Educational Scenarios](https://mlsanity.com/paper/2505.16160/) (2025)
- [EduEVAL-DB: A Role-Based Dataset for Pedagogical Risk Evaluation in Educational Explanations](https://mlsanity.com/paper/2602.15531/) (2026)
- [AIriskEval-edu: New Dataset for Risk Assessment in AI-mediated K-12 Educational Explanations](https://mlsanity.com/paper/2607.01934/) (2026)
- [Shaping Explanations: Semantic Reward Modeling with Encoder-Only Transformers for GRPO](https://mlsanity.com/paper/2509.13081/) (2025)
- [From Correctness to Collaboration: Toward a Human-Centered Framework for Evaluating AI Agent Behavior in Software Engineering](https://mlsanity.com/paper/2512.23844/) (2025)
- [When Verification Hurts: Asymmetric Effects of Multi-Agent Feedback in Logic Proof Tutoring](https://mlsanity.com/paper/2603.27076/) (2026)
- [Confirming Correct, Missing the Rest: LLM Tutoring Agents Struggle Where Feedback Matters Most](https://mlsanity.com/paper/2605.16207/) (2026)
- [Exploring Conversational Design Choices in LLMs for Pedagogical Purposes: Socratic and Narrative Approaches for Improving Instructor's Teaching Practice](https://mlsanity.com/paper/2509.12107/) (2025)
- [PEARL: Training Socratic Tutors with Pedagogically Aligned Reinforcement Learning](https://mlsanity.com/paper/2605.29582/) (2026)
- [The Deliberative Illusion: Diagnosing Factual Attrition and Stance Homogenization in Multi-Agent LLM Deliberation](https://mlsanity.com/paper/2606.03032/) (2026)
- [OnlineMate: An LLM-Based Multi-Agent Companion System for Cognitive Support in Online Learning](https://mlsanity.com/paper/2509.14803/) (2025)
- [CGF-DETR: Cross-Gated Fusion DETR for Enhanced Pneumonia Detection in Chest X-rays](https://mlsanity.com/paper/2511.01730/) (2025)
- [CogEvo-Edu: Cognitive Evolution Educational Multi-Agent Collaborative System](https://mlsanity.com/paper/2512.00331/) (2025)
- [OpenLearnLM Benchmark: A Unified Framework for Evaluating Knowledge, Skill, and Attitude in Educational Large Language Models](https://mlsanity.com/paper/2601.13882/) (2026)
- [TeachBench: A Syllabus-Grounded Framework for Evaluating Teaching Ability in Large Language Models](https://mlsanity.com/paper/2601.21375/) (2026)
- [TeachingCoach: A Fine-Tuned Scaffolding Chatbot for Instructional Guidance to Instructors](https://mlsanity.com/paper/2603.18189/) (2026)
- [Evaluating Epistemic Guardrails in AI Reading Assistants: A Behavioral Audit of a Minimal Prototype](https://mlsanity.com/paper/2604.27275/) (2026)
- [Agentic AI Ecosystems in Higher Education: A Perspective on AI Agents to Emerging Inclusive, Agentic Multi-Agent AI Framework for Learning, Teaching and Institutional Intelligence](https://mlsanity.com/paper/2605.14266/) (2026)
- [Lect=uraAgents: A Multi-Agent Framework for Adaptive Personalized AI-Assisted Learning and Embodied Teaching](https://mlsanity.com/paper/2606.16428/) (2026)
- [The Energy Society: A Simulation Environment for Studying Agent Cooperation under Survival Pressure](https://mlsanity.com/paper/2607.14865/) (2026)
- [ELBench: A Multi-Dimensional Benchmark for Education-Facing Large Language Models](https://mlsanity.com/paper/2608.09548/) (2026)
- [SLATE: Are AI-Generated Slides Educationally Effective? A Benchmark for Language Teaching Quality and Learner Knowledge Acquisition](https://mlsanity.com/paper/2609.06212/) (2026)
- [From Content Generation to Learning Support: Pedagogy-Guided Generative Video Tutors for STEM Learning](https://mlsanity.com/paper/2609.24083/) (2026)

## Abstract (arXiv)

Large language models (LLMs) increasingly serve as educational tools, yet evaluating their teaching capabilities remains challenging due to the resource-intensive, context-dependent, and methodologically complex nature of teacher-student interactions. We introduce EducationQ, a multi-agent dialogue framework that efficiently assesses teaching capabilities through simulated dynamic educational scenarios, featuring specialized agents for teaching, learning, and evaluation. Testing 14 LLMs across major AI Organizations (OpenAI, Meta, Google, Anthropic, and others) on 1,498 questions spanning 13 disciplines and 10 difficulty levels reveals that teaching effectiveness does not correlate linearly with model scale or general reasoning capabilities - with some smaller open-source models outperforming larger commercial counterparts in teaching contexts. This finding highlights a critical gap in current evaluations that prioritize knowledge recall over interactive pedagogy. Our mixed-methods evaluation, combining quantitative metrics with qualitative analysis and expert case studies, identifies distinct pedagogical strengths employed by top-performing models (e.g., sophisticated questioning strategies, adaptive feedback mechanisms). Human expert evaluations show 78% agreement with our automated qualitative analysis of effective teaching behaviors, validating our methodology. EducationQ demonstrates that LLMs-as-teachers require specialized optimization beyond simple scaling, suggesting next-generation educational AI prioritize targeted enhancement of specific pedagogical effectiveness.