# Evaluating Gemini in an arena for learning

arXiv: https://arxiv.org/abs/2505.24477 · HTML: https://mlsanity.com/paper/2505.24477/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 54 references that are in the map.
- First posted on arXiv 30 May 2025.

## Builds on (most cited first, 13 of 13)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://mlsanity.com/paper/2311.16502/) (2023)
- [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://mlsanity.com/paper/2403.04132/) (2024)
- [Humanity's Last Exam](https://mlsanity.com/paper/2501.14249/) (2025)
- [Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation](https://mlsanity.com/paper/2412.03304/) (2024)
- [Towards Responsible Development of Generative AI for Education: An Evaluation-Driven Approach](https://mlsanity.com/paper/2407.12687/) (2024)
- [Tutor CoPilot: A Human-AI Approach for Scaling Real-Time Expertise](https://mlsanity.com/paper/2410.03017/) (2024)
- [LearnLM: Improving Gemini for Learning](https://mlsanity.com/paper/2412.16429/) (2024)
- [Human participants in AI research: Ethics and transparency in practice](https://mlsanity.com/paper/2311.01254/) (2023)

## Built on by (most cited first, 11 of 11)

- [3D-R1: Enhancing Reasoning in 3D VLMs for Unified Scene Understanding](https://mlsanity.com/paper/2507.23478/) (2025)
- [TeachLM: Post-Training LLMs for Education Using Authentic Learning Data](https://mlsanity.com/paper/2510.05087/) (2025)
- [Benchmarking the Pedagogical Knowledge of Large Language Models](https://mlsanity.com/paper/2506.18710/) (2025)
- [AI tutoring can safely and effectively support students: An exploratory RCT in UK classrooms](https://mlsanity.com/paper/2512.23633/) (2025)
- [Architecting Trust in Artificial Epistemic Agents](https://mlsanity.com/paper/2603.02960/) (2026)
- [MRD: Multi-resolution Retrieval-Detection Fusion for High-Resolution Image Understanding](https://mlsanity.com/paper/2512.02906/) (2025)
- [PEARL: Training Socratic Tutors with Pedagogically Aligned Reinforcement Learning](https://mlsanity.com/paper/2605.29582/) (2026)
- [Hey Chat, Can You Teach Me? Structuring Socratic Dialogue for Human Learning in the Wild](https://mlsanity.com/paper/2606.11744/) (2026)
- [EduClaw-Bench: A Long-Horizon Benchmark for Pedagogical LLM Agents with Simulated Learners](https://mlsanity.com/paper/2608.03206/) (2026)
- [Developing a General Personal Tutor for Education](https://mlsanity.com/paper/2512.04869/) (2025)
- [Ceci n'est pas une explication: Evaluating Explanation Failures as Explainability Pitfalls in Language Learning Systems](https://mlsanity.com/paper/2604.26145/) (2026)

## Abstract (arXiv)

Artificial intelligence (AI) is poised to transform education, but the research community lacks a robust, general benchmark to evaluate AI models for learning. To assess state-of-the-art support for educational use cases, we ran an "arena for learning" where educators and pedagogy experts conduct blind, head-to-head, multi-turn comparisons of leading AI models. In particular, N = 189 educators drew from their experience to role-play realistic learning use cases, interacting with two models sequentially, after which N = 206 experts judged which model better supported the user's learning goals. The arena evaluated a slate of state-of-the-art models: Gemini 2.5 Pro, Claude 3.7 Sonnet, GPT-4o, and OpenAI o3. Excluding ties, experts preferred Gemini 2.5 Pro in 73.2% of these match-ups -- ranking it first overall in the arena. Gemini 2.5 Pro also demonstrated markedly higher performance across key principles of good pedagogy. Altogether, these results position Gemini 2.5 Pro as a leading model for learning.