# EduBench: A Comprehensive Benchmarking Dataset for Evaluating Large Language Models in Diverse Educational Scenarios

arXiv: https://arxiv.org/abs/2505.16160 · HTML: https://mlsanity.com/paper/2505.16160/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 41 references that are in the map.
- First posted on arXiv 22 May 2025.

## Builds on (most cited first, 24 of 24)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [Inference-Time Scaling for Generalist Reward Modeling](https://mlsanity.com/paper/2504.02495/) (2025)
- [Large Language Models for Education: A Survey and Outlook](https://mlsanity.com/paper/2403.18105/) (2024)
- [Simulating Classroom Education with LLM-Empowered Agents](https://mlsanity.com/paper/2406.19226/) (2024)
- [Experiences from Using Code Explanations Generated by Large Language Models in a Web Software Development E-Book](https://mlsanity.com/paper/2211.02265/) (2022)
- [Large Language Models in Education: Vision and Opportunities](https://mlsanity.com/paper/2311.13160/) (2023)
- [Learning gain differences between ChatGPT and human tutor generated algebra hints](https://mlsanity.com/paper/2302.06871/) (2023)
- [LLMs can Find Mathematical Reasoning Mistakes by Pedagogical Chain-of-Thought](https://mlsanity.com/paper/2405.06705/) (2024)
- [EducationQ: Evaluating LLMs' Teaching Capabilities Through Multi-Agent Dialogue Framework](https://mlsanity.com/paper/2504.14928/) (2025)
- [Question Generation for Reading Comprehension Assessment by Modeling How and What to Ask](https://mlsanity.com/paper/2204.02908/) (2022)
- [CS1-LLM: Integrating LLMs into CS1 Instruction](https://mlsanity.com/paper/2406.15379/) (2024)
- [EQG-RACE: Examination-Type Question Generation](https://mlsanity.com/paper/2012.06106/) (2020)
- [CREF: An LLM-based Conversational Software Repair Framework for Programming Tutors](https://mlsanity.com/paper/2406.13972/) (2024)
- [Evaluating Contextually Personalized Programming Exercises Created with Generative AI](https://mlsanity.com/paper/2407.11994/) (2024)
- [Generating Educational Materials with Different Levels of Readability using LLMs](https://mlsanity.com/paper/2406.12787/) (2024)
- [Enhancing Computer Programming Education with LLMs: A Study on Effective Prompt Engineering for Python Code Generation](https://mlsanity.com/paper/2407.05437/) (2024)
- [Generative AI for Enhancing Active Learning in Education: A Comparative Study of GPT-3.5 and GPT-4 in Crafting Customized Test Questions](https://mlsanity.com/paper/2406.13903/) (2024)
- [Can Language Models Evaluate Human Written Text? Case Study on Korean Student Writing for Education](https://mlsanity.com/paper/2407.17022/) (2024)
- [A Multi-language Platform for Generating Algebraic Mathematical Word Problems](https://mlsanity.com/paper/1912.01110/) (2019)

## Built on by (most cited first, 16 of 16)

- [Opportunities and Challenges of LLMs in Education: An NLP Perspective](https://mlsanity.com/paper/2507.22753/) (2025)
- [EduGuardBench: A Holistic Benchmark for Evaluating the Pedagogical Fidelity and Adversarial Safety of LLMs as Simulated Teachers](https://mlsanity.com/paper/2511.06890/) (2025)
- [Cultivating Helpful, Personalized, and Creative AI Tutors: A Framework for Pedagogical Alignment using Reinforcement Learning](https://mlsanity.com/paper/2507.20335/) (2025)
- [From Answers to Questions: EQGBench for Evaluating LLMs' Educational Question Generation](https://mlsanity.com/paper/2508.10005/) (2025)
- [OmniEduBench: A Comprehensive Chinese Benchmark for Evaluating Large Language Models in Education](https://mlsanity.com/paper/2510.26422/) (2025)
- [MENTOR: A Metacognition-Driven Self-Evolution Framework for Uncovering and Mitigating Implicit Domain Risks in LLMs](https://mlsanity.com/paper/2511.07107/) (2025)
- [Rethinking Scaffolding in LLM Tutors: The Interactional Mismatch Between Benchmarks and Real-World Deployments](https://mlsanity.com/paper/2606.15766/) (2026)
- [Bringing Multi-Modal Multi-Task Federated Foundation Models to Education Domain: Prospects and Challenges](https://mlsanity.com/paper/2509.07946/) (2025)
- [OpenLearnLM Benchmark: A Unified Framework for Evaluating Knowledge, Skill, and Attitude in Educational Large Language Models](https://mlsanity.com/paper/2601.13882/) (2026)
- [EduResearchBench: A Hierarchical Atomic Task Decomposition Benchmark for Full-Lifecycle Educational Research](https://mlsanity.com/paper/2602.15034/) (2026)
- [Benchmarking Local Language Models for Social Robots using Edge Devices](https://mlsanity.com/paper/2605.03111/) (2026)
- [Are Agents Ready to Teach? A Multi-Stage Benchmark for Real-World Teaching Workflows](https://mlsanity.com/paper/2605.14322/) (2026)
- [CSTutorBench: Benchmarking Small Language Models as Tutors for Block-Based Programming](https://mlsanity.com/paper/2607.05571/) (2026)
- [ELBench: A Multi-Dimensional Benchmark for Education-Facing Large Language Models](https://mlsanity.com/paper/2608.09548/) (2026)
- [ADE: Agentic Data Evolution Framework for Human-Centered Objectives](https://mlsanity.com/paper/2608.23719/) (2026)
- [The Architect, the Adversary, and the Judge: Closed-Loop Generation of Standards-Aligned Assessment Items at Scale](https://mlsanity.com/paper/2609.26087/) (2026)

## Abstract (arXiv)

As large language models continue to advance, their application in educational contexts remains underexplored and under-optimized. In this paper, we address this gap by introducing the first diverse benchmark tailored for educational scenarios, incorporating synthetic data containing 9 major scenarios and over 4,000 distinct educational contexts. To enable comprehensive assessment, we propose a set of multi-dimensional evaluation metrics that cover 12 critical aspects relevant to both teachers and students. We further apply human annotation to ensure the effectiveness of the model-generated evaluation responses. Additionally, we succeed to train a relatively small-scale model on our constructed dataset and demonstrate that it can achieve performance comparable to state-of-the-art large models (e.g., Deepseek V3, Qwen Max) on the test set. Overall, this work provides a practical foundation for the development and evaluation of education-oriented language models. Code and data are released at https://github.com/ybai-nlp/EduBench.