# Findings of the First Teaching Monster Challenge: A Benchmark of Pedagogical Content Knowledge in AI Agents

arXiv: https://arxiv.org/abs/2608.08852 · HTML: https://mlsanity.com/paper/2608.08852/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Educational agents and math problem solving, one of 16 directions in LLM agents and tool use.
- Builds on 15 of its 28 references that are in the map.
- First posted on arXiv 9 Aug 2026.

## Placed in

- LLM agents and tool use / [Educational agents and math problem solving](https://mlsanity.com/agents/educational-agents-and-math-problem-solving/)

## Builds on (most cited first, 15 of 15)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://mlsanity.com/paper/2403.04132/) (2024)
- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [EvalCrafter: Benchmarking and Evaluating Large Video Generation Models](https://mlsanity.com/paper/2310.11440/) (2023)
- [PPTAgent: Generating and Evaluating Presentations Beyond Text-to-Slides](https://mlsanity.com/paper/2501.03936/) (2025)
- [GenAI Arena: An Open Evaluation Platform for Generative Models](https://mlsanity.com/paper/2406.04485/) (2024)
- [DOC2PPT: Automatic Presentation Slides Generation from Scientific Documents](https://mlsanity.com/paper/2101.11796/) (2021)
- [MathTutorBench: A Benchmark for Measuring Open-ended Pedagogical Capabilities of LLM Tutors](https://mlsanity.com/paper/2502.18940/) (2025)
- [The BEA 2023 Shared Task on Generating AI Teacher Responses in Educational Dialogues](https://mlsanity.com/paper/2306.06941/) (2023)
- [Code2Video: A Code-centric Paradigm for Educational Video Generation](https://mlsanity.com/paper/2510.01174/) (2025)
- [TheoremExplainAgent: Towards Video-based Multimodal Explanations for LLM Theorem Understanding](https://mlsanity.com/paper/2502.19400/) (2025)
- [Enhancing Presentation Slide Generation by LLMs with a Multi-Staged End-to-End Approach](https://mlsanity.com/paper/2406.06556/) (2024)

## Built on by (most cited first, 1 of 1)

- [Reels-RSI: Program-Level Self-Improvement for Code-Rendered Explainer Videos](https://mlsanity.com/paper/2610.14495/) (2026)

## Abstract (arXiv)

AI agents can now solve problems, answer like subject experts, and generate long-form multimodal content. However, whether they can adapt a lesson to fit a specified learner, which education calls Pedagogical Content Knowledge (PCK), has not been benchmarked. To measure it, we introduce the Teaching Monster Challenge, the first instructional video generation benchmark to treat the learner persona as an explicit evaluation criterion. Each system is given a topic and a learner persona and must generate a complete instructional video. Every video is screened by an LLM-judge, ranked by crowd pairwise voting, and finalized by an expert panel. The first edition shows that today's systems handle the content well but are far weaker at presenting it and adapting it to the learner. The same process exposes a limit of automatic judging. The LLM-judge separates a clear low-performing tail but ranks the strongest systems poorly. The strongest systems receive nearly identical scores from the judge, so its ranking of them does not match human preference. Progress therefore requires not only better teaching systems but also better automatic judges, and we release the benchmark, rubric, and human judgments as a testbed for both.