mlsanityData to 7 Oct 2026

EduPanel: A Three-Agent LLM Judge for Teaching Videos -- Reliability, Complementarity, and Human Trust Calibration

arXiv 2607.18529 ↗First posted 20 Jul 2026cs.HC cs.AIcited by 0

On the map: LLM agents and tool use → Educational agents and math problem solving

Abstract (arXiv)

Teaching videos are becoming a major medium for education, creating a growing need for scalable evaluation of their pedagogical quality. Existing automatic judges do not fully address this setting because teaching quality depends on multimodal evidence and should be evaluated with respect to the intended learner rather than as a universal property. We present EduPanel, a rubric-grounded, learner-conditioned LLM judge that decomposes evaluation across specialized agents to produce interpretable assessments for different aspects of teaching quality. Across expert studies, architecture ablations, and learner-persona analyses, EduPanel achieves reliability comparable to a median human expert. In expert evaluation, its feedback improves scoring accuracy (MAE 0.87 to 0.73), while experts remain able to detect unreliable outputs (AUC = 0.77) instead of accepting them blindly. These results suggest that EduPanel can serve as effective assistants for educational evaluation rather than replacements for human experts.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Educational agents and math problem solving, one of 16 directions in LLM agents and tool use. Builds on 23 of its 44 references that are in the map. First posted on arXiv 20 Jul 2026.

Builds on

23 of 44 references.

References · 23

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  2. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  3. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  4. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  5. 2024A Survey on LLM-as-a-Judgecited by 1,384
  6. 2023Large Language Models are not Fair Evaluatorscited by 1,050
  7. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 865
  8. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
  9. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 651
  10. 2023Prometheus: Inducing Fine-grained Evaluation Capability in Language Modelscited by 558
  11. 2024RewardBench: Evaluating Reward Models for Language Modelingcited by 480
  12. 2020Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performancecited by 396
11 more
  1. 2024MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmarkcited by 362
  2. 2024LLaVA-Critic: Learning to Evaluate Multimodal Modelscited by 132
  3. 2024Generative Students: Using LLM-Simulated Student Profiles to Support Question Item Evaluationcited by 56
  4. 2023Large Language Models Help Humans Verify Truthfulness -- Except When They Are Convincingly Wrongcited by 51
  5. 2023Is ChatGPT a Good Teacher Coach? Measuring Zero-Shot Performance For Scoring and Providing Actionable Insights on Classroom Instructioncited by 37
  6. 2024Video-Language Understanding: A Survey from Model Architecture, Model Training, and Data Perspectivescited by 37
  7. 2025Code2Video: A Code-centric Paradigm for Educational Video Generationcited by 23
  8. 2025Paper2Video: Automatic Video Generation from Scientific Paperscited by 23
  9. 2022Pre-Avatar: An Automatic Presentation Generation Framework Leveraging Talking Avatarcited by 6
  10. 2025Simulating Students with Large Language Models: A Review of Architecture, Mechanisms, and Role Modelling in Education with Generative AIcited by 5
  11. 2025Generating Narrated Lecture Videos from Slides with Synchronized Highlightscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Findings of the First Teaching Monster Challenge: A Benchmark of Pedagogical Content Knowledge in AI Agents
  2. 2025Conversational Education at Scale: A Multi-LLM Agent Workflow for Procedural Learning and Pedagogic Quality Assessment
  3. 2025Dean of LLM Tutors: A Framework for Automated Quality Review of AI-generated Feedback
  4. 2026RTLC -- Research, Teach-to-Learn, Critique: A three-stage prompting paradigm inspired by the Feynman Learning Technique that lifts LLM-as-judge accuracy on JudgeBench with no fine-tuning
  5. 2025Multi-Agent-as-Judge: Aligning LLM-Agent-Based Automated Evaluation with Multi-Dimensional Human Evaluation
  6. 2026Evidence-Decision-Feedback: Theory-Driven Adaptive Scaffolding for LLM Agents
  7. 2026EduClaw-Bench: A Long-Horizon Benchmark for Pedagogical LLM Agents with Simulated Learners
  8. 2026FairJudge: An Adaptive, Debiased, and Consistent LLM-as-a-Judge