mlsanityData to 7 Oct 2026

The Aftermath of DrawEduMath: Vision Language Models Underperform with Struggling Students and Misdiagnose Errors

arXiv 2603.00925 ↗First posted 1 Mar 2026cs.CL cs.CV cs.CYcited by 1

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Effective mathematics education requires identifying and responding to students' mistakes. For AI to support pedagogical applications, models must perform well across different levels of student proficiency. Our work provides an extensive, year-long snapshot of how 11 vision-language models (VLMs) perform on DrawEduMath, a QA benchmark involving real students' handwritten, hand-drawn responses to math problems. We find that models' weaknesses concentrate on a core component of math education: student error. All evaluated VLMs underperform when describing work from students who require more pedagogical help, and across all QA, they struggle the most on questions related to assessing student error. Thus, while VLMs may be optimized to be math problem solving experts, our results suggest that they require alternative development incentives to adequately support educational use cases.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 30 of its 63 references that are in the map. First posted on arXiv 1 Mar 2026.

Builds on

30 of 63 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025VEHME: A Vision-Language Model For Evaluating Handwritten Mathematics Expressions26 Oct
  2. 2025DrawEduMath: Evaluating Vision Language Models with Expert-Annotated Students' Hand-Drawn Math Images24 Jan
  3. 2025Can Vision-Language Models Evaluate Handwritten Math?13 Jan
  4. 2024A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challenges16 Dec
  5. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?21 Mar
  6. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct

Other references · 24

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  3. 2020Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditingcited by 380
  4. 2023A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicitycited by 180
  5. 2023OpenWebMath: An Open Dataset of High-Quality Mathematical Web Textcited by 151
  6. 2022The Fallacy of AI Functionalitycited by 88
  7. 2022CREPE: Open-Domain Question Answering with False Presuppositionscited by 70
  8. 2024Exploring LLM Prompting Strategies for Joint Essay Scoring and Feedback Generationcited by 63
  9. 2024POSIX: A Prompt Sensitivity Index For Large Language Modelscited by 56
  10. 2024Stepwise Verification and Remediation of Student Reasoning Errors with Large Language Model Tutorscited by 39
  11. 2025Nemotron-CC-Math: A 133 Billion-Token-Scale High Quality Math Pretraining Datasetcited by 34
  12. 2024Reasoning Robustness of LLMs to Adversarial Typographical Errorscited by 31
12 more
  1. 2022Towards Automated Document Revision: Grammatical Error Correction, Fluency Edits, and Beyondcited by 20
  2. 2025Learning to Make MISTAKEs: Modeling Incorrect Student Thinking And Key Errorscited by 19
  3. 2025Findings of the BEA 2025 Shared Task on Pedagogical Ability Assessment of AI-powered Tutorscited by 16
  4. 2025"Don't Forget the Teachers": Towards an Educator-Centered Understanding of Harms from Large Language Models in Educationcited by 14
  5. 2023Pregnant Questions: The Importance of Pragmatic Awareness in Maternal Health Question Answeringcited by 13
  6. 2021Equity and Artificial Intelligence in Education: Will "AIEd" Amplify or Alleviate Inequities in Education?cited by 9
  7. 2025Interactive Sketchpad: A Multimodal Tutoring System for Collaborative, Visual Problem-Solvingcited by 9
  8. 2025LLMs cannot spot math errors, even when allowed to peek into the solutioncited by 7
  9. 2021Risks of AI Foundation Models in Educationcited by 6
  10. 2025SketchMind: A Multi-Agent Cognitive Framework for Assessing Student-Drawn Scientific Sketchescited by 3
  11. 2025Teaching Models to Understand (but not Generate) High-risk Datacited by 2
  12. 2025Dead Zone of Accountability: Why Social Claims in Machine Learning Research Should Be Articulated and Defendedcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026M3-ACE: Rectifying Visual Perception in Multimodal Math Reasoning via Multi-Agentic Context Engineeringcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025MathEDU: Feedback Generation on Problem-Solving Processes for Mathematical Learning Support
  2. 2025Mathematical Computation and Reasoning Errors by Large Language Models
  3. 2025Benchmarking Large Language Models for Diagnosing Students' Cognitive Skills from Handwritten Math Work
  4. 2026Exploring Agentic Workflows for Generating High Quality Math Visual Aids
  5. 2024DynaMath: A Dynamic Visual Benchmark for Evaluating Mathematical Reasoning Robustness of Vision Language Models
  6. 2026Can Vision Language Models Be Adaptive in Mathematics Education? A Learner Model-based Rubric Study
  7. 2025Seeing the Big Picture: Evaluating Multimodal LLMs' Ability to Interpret and Grade Handwritten Student Work
  8. 2026When VLMs 'Fix' Students: Identifying and Penalizing Over-Correction in the Evaluation of Multi-line Handwritten Math OCR