mlsanityData to 7 Oct 2026

LiveK12Bench: Have Large Multimodal Models Truly Conquered High School-level Examinations?

arXiv 2605.26781 ↗First posted 26 May 2026cs.AI cs.MMcited by 3

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Advanced Large Multimodal Models (LMMs) have demonstrated impressive performance in K-12 reasoning tasks, exhibiting great promise as intelligent tutors. Realizing this potential requires models to navigate real-world examinations effectively, yet most existing benchmarks fail to capture the complexity of authentic testing environments. Specifically, most datasets are static, prone to data contamination, and are often confined to restricted modalities, disciplines, and evaluation criteria. To address these issues, we introduce LiveK12Bench, a dynamic, holistic, multi-disciplinary benchmark designed to evaluate the reasoning abilities of LMMs in realistic examination scenarios. LiveK12Bench comprises 2K+ verified questions spanning Mathematics, Physics, Chemistry, and Biology, sourced from the latest real-world exam papers and designed to grow over time. Our framework features several core innovations: 1) featuring an automated pipeline that continuously ingests and parses the latest examination papers to mitigate data leakage; and 2) proposing a novel `Mock Exam' evaluation scheme, which assesses the ability to complete end-to-end exams autonomously with accurate and efficient reasoning paths. Extensive experiments on 12 LMMs reveal that advanced models suffer substantial performance degradation under exam-realistic constraints: GPT-5's score drops from 79 to 53 (out of 100) when process rigor and efficiency are jointly evaluated. Our findings expose critical vulnerabilities, such as sensitivity to complex visual layouts, highlighting the gap between idealized reasoning capabilities and true educational readiness. Both code and dataset are publicly available.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 24 of its 37 references that are in the map. First posted on arXiv 26 May 2026.

Builds on

24 of 37 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Qwen3-VL Technical Report26 Nov
  2. 2025K12Vista: Exploring the Boundaries of MLLMs in K-12 Education2 Jun
  3. 2025MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models8 Apr

Other references · 21

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  4. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  5. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  6. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 573
  7. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 516
  8. 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 502
  9. 2026Kimi K2.5: Visual Agentic Intelligencecited by 449
  10. 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 339
  11. 2024We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?cited by 281
  12. 2025O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruningcited by 278
9 more
  1. 2024MinerU: An Open-Source Solution for Precise Document Content Extractioncited by 255
  2. 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Modelscited by 246
  3. 2023Evaluating the Performance of Large Language Models on GAOKAO Benchmarkcited by 182
  4. 2023M3Exam: A Multilingual, Multimodal, Multilevel Benchmark for Examining Large Language Modelscited by 109
  5. 2024EXAMS-V: A Multi-Discipline Multilingual Multimodal Exam Benchmark for Evaluating Vision Language Modelscited by 41
  6. 2025We-Math 2.0: A Versatile MathBook System for Incentivizing Visual Mathematical Reasoningcited by 32
  7. 2023GRASP: A novel benchmark for evaluating language GRounding And Situated Physics understanding in multimodal language modelscited by 28
  8. 2024GAOKAO-MM: A Chinese Human-Level Benchmark for Multimodal Models Evaluationcited by 10
  9. 2025MMSciBench: Benchmarking Language Models on Chinese Multimodal Scientific Problemscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025MDK12-Bench: A Comprehensive Evaluation of Multimodal Large Language Models on Multidisciplinary Exams
  2. 2026Human-Grounded Multimodal Benchmark with 900K-Scale Aggregated Student Response Distributions from Japan's National Assessment of Academic Ability
  3. 2024VisScience: An Extensive Benchmark for Evaluating K12 Educational Multi-modal Scientific Reasoning
  4. 2026Visual Reasoning Benchmark: Evaluating Multimodal LLMs on Classroom-Authentic Visual Problems from Primary Education
  5. 2024SceMQA: A Scientific College Entrance Level Multimodal Question Answering Benchmark
  6. 2025BMMR: A Large-Scale Bilingual Multimodal Multi-Discipline Reasoning Dataset
  7. 2025MathReal: We Keep It Real! A Real Scene Benchmark for Evaluating Math Reasoning in Multimodal Large Language Models
  8. 2025Closing the Gap: Data-Centric Fine-Tuning of Vision Language Models for the Standardized Exam Questions