# Towards Responsible Development of Generative AI for Education: An Evaluation-Driven Approach

arXiv: https://arxiv.org/abs/2407.12687 · HTML: https://mlsanity.com/paper/2407.12687/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 May 2024.

## Built on by (most cited first, 30 of 54)

- [Unifying AI Tutor Evaluation: An Evaluation Taxonomy for Pedagogical Ability Assessment of LLM-Powered AI Tutors](https://mlsanity.com/paper/2412.09416/) (2024)
- [Tutor CoPilot: A Human-AI Approach for Scaling Real-Time Expertise](https://mlsanity.com/paper/2410.03017/) (2024)
- [Stepwise Verification and Remediation of Student Reasoning Errors with Large Language Model Tutors](https://mlsanity.com/paper/2407.09136/) (2024)
- [ChatBench: From Static Benchmarks to Human-AI Evaluation](https://mlsanity.com/paper/2504.07114/) (2025)
- [TeachTune: Reviewing Pedagogical Agents Against Diverse Student Profiles with Simulated Students](https://mlsanity.com/paper/2410.04078/) (2024)
- [LearnLM: Improving Gemini for Learning](https://mlsanity.com/paper/2412.16429/) (2024)
- [Towards the Pedagogical Steering of Large Language Models for Tutoring: A Case Study with Modeling Productive Failure](https://mlsanity.com/paper/2410.03781/) (2024)
- [Evaluating Gemini in an arena for learning](https://mlsanity.com/paper/2505.24477/) (2025)
- [Help Me Write a Story: Evaluating LLMs' Ability to Generate Writing Feedback](https://mlsanity.com/paper/2507.16007/) (2025)
- [Comparative Analysis Based on DeepSeek, ChatGPT, and Google Gemini: Features, Techniques, Performance, Future Prospects](https://mlsanity.com/paper/2503.04783/) (2025)
- [SPHERE: An Evaluation Card for Human-AI Systems](https://mlsanity.com/paper/2504.07971/) (2025)
- [TeachLM: Post-Training LLMs for Education Using Authentic Learning Data](https://mlsanity.com/paper/2510.05087/) (2025)
- [DiVERT: Distractor Generation with Variational Errors Represented as Text for Math Multiple-choice Questions](https://mlsanity.com/paper/2406.19356/) (2024)
- [Personalizing Student-Agent Interactions Using Log-Contextualized Retrieval-Augmented Generation (RAG)](https://mlsanity.com/paper/2505.17238/) (2025)
- [Mathfish: Evaluating Language Model Math Reasoning via Grounding in Educational Curricula](https://mlsanity.com/paper/2408.04226/) (2024)
- [Benchmarking the Pedagogical Knowledge of Large Language Models](https://mlsanity.com/paper/2506.18710/) (2025)
- [Pedagogy-driven Evaluation of Generative AI-powered Intelligent Tutoring Systems](https://mlsanity.com/paper/2510.22581/) (2025)
- [ELI-Why: Evaluating the Pedagogical Utility of Language Model Explanations](https://mlsanity.com/paper/2506.14200/) (2025)
- [PATS: Personality-Aware Teaching Strategies with Large Language Model Tutors](https://mlsanity.com/paper/2601.08402/) (2026)
- [The Life Cycle of Large Language Models: A Review of Biases in Education](https://mlsanity.com/paper/2407.11203/) (2024)
- [AI tutoring can safely and effectively support students: An exploratory RCT in UK classrooms](https://mlsanity.com/paper/2512.23633/) (2025)
- [Evidence-Decision-Feedback: Theory-Driven Adaptive Scaffolding for LLM Agents](https://mlsanity.com/paper/2602.01415/) (2026)
- [Closing the Loop: Learning to Generate Writing Feedback via Language Model Simulated Student Revisions](https://mlsanity.com/paper/2410.08058/) (2024)
- ["Would You Want an AI Tutor?" Understanding Stakeholder Perceptions of LLM-based Systems in the Classroom](https://mlsanity.com/paper/2503.02885/) (2025)
- [A Framework for Situating Innovations, Opportunities, and Challenges in Advancing Vertical Systems with Large AI Models](https://mlsanity.com/paper/2504.02793/) (2025)
- [From Correctness to Collaboration: Toward a Human-Centered Framework for Evaluating AI Agent Behavior in Software Engineering](https://mlsanity.com/paper/2512.23844/) (2025)
- [AgentTutor: Empowering Personalized Learning with Multi-Turn Interactive Teaching in Intelligent Education Systems](https://mlsanity.com/paper/2601.04219/) (2025)
- [Knowledge without Wisdom: Measuring Misalignment between LLMs and Intended Impact](https://mlsanity.com/paper/2603.00883/) (2026)
- [From Intuition to Understanding: Using AI Peers to Overcome Physics Misconceptions](https://mlsanity.com/paper/2504.00408/) (2025)
- [Enhanced Question-Answering for Skill-based learning using Knowledge-based AI and Generative AI](https://mlsanity.com/paper/2504.07463/) (2025)

## Abstract (arXiv)

A major challenge facing the world is the provision of equitable and universal access to quality education. Recent advances in generative AI (gen AI) have created excitement about the potential of new technologies to offer a personal tutor for every learner and a teaching assistant for every teacher. The full extent of this dream, however, has not yet materialised. We argue that this is primarily due to the difficulties with verbalising pedagogical intuitions into gen AI prompts and the lack of good evaluation practices, reinforced by the challenges in defining excellent pedagogy. Here we present our work collaborating with learners and educators to translate high level principles from learning science into a pragmatic set of seven diverse educational benchmarks, spanning quantitative, qualitative, automatic and human evaluations; and to develop a new set of fine-tuning datasets to improve the pedagogical capabilities of Gemini, introducing LearnLM-Tutor. Our evaluations show that LearnLM-Tutor is consistently preferred over a prompt tuned Gemini by educators and learners on a number of pedagogical dimensions. We hope that this work can serve as a first step towards developing a comprehensive educational evaluation framework, and that this can enable rapid progress within the AI and EdTech communities towards maximising the positive impact of gen AI in education.