# AIriskEval-edu: New Dataset for Risk Assessment in AI-mediated K-12 Educational Explanations

arXiv: https://arxiv.org/abs/2607.01934 · HTML: https://mlsanity.com/paper/2607.01934/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 33 references that are in the map.
- First posted on arXiv 2 Jul 2026.

## Builds on (most cited first, 18 of 18)

- [Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering](https://mlsanity.com/paper/2209.09513/) (2022)
- [A General Language Assistant as a Laboratory for Alignment](https://mlsanity.com/paper/2112.00861/) (2021)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models](https://mlsanity.com/paper/2309.01219/) (2023)
- [MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems](https://mlsanity.com/paper/2305.14536/) (2023)
- [Unifying AI Tutor Evaluation: An Evaluation Taxonomy for Pedagogical Ability Assessment of LLM-Powered AI Tutors](https://mlsanity.com/paper/2412.09416/) (2024)
- [LLMs are Biased Teachers: Evaluating LLM Bias in Personalized Education](https://mlsanity.com/paper/2410.14012/) (2024)
- [EducationQ: Evaluating LLMs' Teaching Capabilities Through Multi-Agent Dialogue Framework](https://mlsanity.com/paper/2504.14928/) (2025)
- [edBB-Demo: Biometrics and Behavior Analysis for Online Educational Platforms](https://mlsanity.com/paper/2211.09210/) (2022)
- [Findings of the BEA 2025 Shared Task on Pedagogical Ability Assessment of AI-powered Tutors](https://mlsanity.com/paper/2507.10579/) (2025)
- [Boosting Large Language Models with Socratic Method for Conversational Mathematics Teaching](https://mlsanity.com/paper/2407.17349/) (2024)
- [E-EVAL: A Comprehensive Chinese K-12 Education Evaluation Benchmark for Large Language Models](https://mlsanity.com/paper/2401.15927/) (2024)
- [Addressing Bias in LLMs: Strategies and Application to Fair AI-based Recruitment](https://mlsanity.com/paper/2506.11880/) (2025)
- [Biometrics and Behavior Analysis for Detecting Distractions in e-Learning](https://mlsanity.com/paper/2405.15434/) (2024)
- [Is My Vision-Language Data in Your AI? Membership Inference Test (MINT) Demo 2](https://mlsanity.com/paper/2606.14748/) (2026)
- [AI-based Multimodal Biometrics for Detecting Smartphone Distractions: Application to Online Learning](https://mlsanity.com/paper/2506.17364/) (2025)
- [Evaluating Social Engineering Risks in AI-based Interaction using Biometrics and a Gaming Setup](https://mlsanity.com/paper/2606.17793/) (2026)
- [EduEVAL-DB: A Role-Based Dataset for Pedagogical Risk Evaluation in Educational Explanations](https://mlsanity.com/paper/2602.15531/) (2026)

## Built on by (most cited first, 3 of 3)

- [Comparative Study of Domain-adapted VLMs for General Document Visual Question Answering](https://mlsanity.com/paper/2607.07179/) (2026)
- [AIriskEval-edu Demo: Auditing of Pedagogical Risks in Educational Explanations](https://mlsanity.com/paper/2607.25634/) (2026)
- [inLoRA-MINT: Internal Representations for Auditing Training Data in Domain-adapted LLMs](https://mlsanity.com/paper/2610.18051/) (2026)

## Abstract (arXiv)

This work introduces AIriskEval-edu-db2, a new dataset designed to train and evaluate auditors based on LLMs for an explainable pedagogical risk assessment in instructional content for grades K-12. The dataset comprises 1,639 explanations from 170 curated ScienceQA questions, covering science, language arts, and social sciences. For each question, the dataset includes an explanation written by a human teacher alongside 11 explanations generated by LLM-simulated teacher profiles associated with distinct pedagogical risks. We propose a comprehensive risk rubric aligned with established educational standards that covers five complementary dimensions: factual precision, depth and completeness, focus and relevance, student-level appropriateness, and ideological bias. A key contribution is the addition of 785 explanations with structured explainability annotations, including risk localization and risk description. The annotations are produced through a semi-automatic process with expert teacher validation. Finally, we present validation experiments comparing state-of-the-art proprietary models with a lightweight local Llama 3.1 8B model in both the pedagogical risk detection and the explainability assessment. These experiments evaluate whether supervised fine-tuning on AIriskEval-edu-db2 enables a locally deployable model to approach or outperform stronger frontier models while preserving privacy in educational auditing and assessment tasks.