# EduEVAL-DB: A Role-Based Dataset for Pedagogical Risk Evaluation in Educational Explanations

arXiv: https://arxiv.org/abs/2602.15531 · HTML: https://mlsanity.com/paper/2602.15531/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 38 references that are in the map.
- First posted on arXiv 17 Feb 2026.

## Builds on (most cited first, 19 of 19)

- [Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering](https://mlsanity.com/paper/2209.09513/) (2022)
- [A General Language Assistant as a Laboratory for Alignment](https://mlsanity.com/paper/2112.00861/) (2021)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models](https://mlsanity.com/paper/2309.01219/) (2023)
- [MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems](https://mlsanity.com/paper/2305.14536/) (2023)
- [Unifying AI Tutor Evaluation: An Evaluation Taxonomy for Pedagogical Ability Assessment of LLM-Powered AI Tutors](https://mlsanity.com/paper/2412.09416/) (2024)
- [MathTutorBench: A Benchmark for Measuring Open-ended Pedagogical Capabilities of LLM Tutors](https://mlsanity.com/paper/2502.18940/) (2025)
- [LLMs are Biased Teachers: Evaluating LLM Bias in Personalized Education](https://mlsanity.com/paper/2410.14012/) (2024)
- [EducationQ: Evaluating LLMs' Teaching Capabilities Through Multi-Agent Dialogue Framework](https://mlsanity.com/paper/2504.14928/) (2025)
- [edBB-Demo: Biometrics and Behavior Analysis for Online Educational Platforms](https://mlsanity.com/paper/2211.09210/) (2022)
- [Findings of the BEA 2025 Shared Task on Pedagogical Ability Assessment of AI-powered Tutors](https://mlsanity.com/paper/2507.10579/) (2025)
- [Boosting Large Language Models with Socratic Method for Conversational Mathematics Teaching](https://mlsanity.com/paper/2407.17349/) (2024)
- [E-EVAL: A Comprehensive Chinese K-12 Education Evaluation Benchmark for Large Language Models](https://mlsanity.com/paper/2401.15927/) (2024)
- [A multimodal dataset for understanding the impact of mobile phones on remote online virtual education](https://mlsanity.com/paper/2412.14195/) (2024)
- [Biometrics and Behavior Analysis for Detecting Distractions in e-Learning](https://mlsanity.com/paper/2405.15434/) (2024)
- [mEBAL2 Database and Benchmark: Image-based Multispectral Eyeblink Detection](https://mlsanity.com/paper/2309.07880/) (2023)
- [DeepFace-Attention: Multimodal Face Biometrics for Attention Estimation with Application to e-Learning](https://mlsanity.com/paper/2408.05523/) (2024)
- [SMARTe-VR: Student Monitoring and Adaptive Response Technology for e-Learning in Virtual Reality](https://mlsanity.com/paper/2501.10977/) (2025)
- [AI-based Multimodal Biometrics for Detecting Smartphone Distractions: Application to Online Learning](https://mlsanity.com/paper/2506.17364/) (2025)

## Built on by (most cited first, 4 of 4)

- [Auditing Training Data in Domain-adapted LLMs: LoRA-MINT](https://mlsanity.com/paper/2606.06946/) (2026)
- [Overview of Risk Assessment and Management for Intelligent Systems under the AI Act and Beyond](https://mlsanity.com/paper/2607.02197/) (2026)
- [AIriskEval-edu: New Dataset for Risk Assessment in AI-mediated K-12 Educational Explanations](https://mlsanity.com/paper/2607.01934/) (2026)
- [AIriskEval-edu Demo: Auditing of Pedagogical Risks in Educational Explanations](https://mlsanity.com/paper/2607.25634/) (2026)

## Abstract (arXiv)

This work introduces EduEVAL-DB, a dataset based on teacher roles designed to support the evaluation and training of automatic pedagogical evaluators and AI tutors for instructional explanations. The dataset comprises 854 explanations corresponding to 139 questions from a curated subset of the ScienceQA benchmark, spanning science, language, and social science across K-12 grade levels. For each question, one human-teacher explanation is provided and six are generated by LLM-simulated teacher roles. These roles are inspired by instructional styles and shortcomings observed in real educational practice and are instantiated via prompt engineering. We further propose a pedagogical risk rubric aligned with established educational standards, operationalizing five complementary risk dimensions: factual correctness, explanatory depth and completeness, focus and relevance, student-level appropriateness, and ideological bias. All explanations are annotated with binary risk labels through a semi-automatic process with expert teacher review. Finally, we present preliminary validation experiments to assess the suitability of EduEVAL-DB for evaluation. We benchmark a state-of-the-art education-oriented model (Gemini 2.5 Pro) against a lightweight local Llama 3.1 8B model and examine whether supervised fine-tuning on EduEVAL-DB supports pedagogical risk detection using models deployable on consumer hardware.