# The Aftermath of DrawEduMath: Vision Language Models Underperform with Struggling Students and Misdiagnose Errors

arXiv: https://arxiv.org/abs/2603.00925 · HTML: https://mlsanity.com/paper/2603.00925/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 30 of its 63 references that are in the map.
- First posted on arXiv 1 Mar 2026.

## Placed in

- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 30 of 30)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023)
- [MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?](https://mlsanity.com/paper/2403.14624/) (2024)
- [Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing](https://mlsanity.com/paper/2001.00973/) (2020)
- [A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity](https://mlsanity.com/paper/2305.13169/) (2023)
- [OpenWebMath: An Open Dataset of High-Quality Mathematical Web Text](https://mlsanity.com/paper/2310.06786/) (2023)
- [The Fallacy of AI Functionality](https://mlsanity.com/paper/2206.09511/) (2022)
- [CREPE: Open-Domain Question Answering with False Presuppositions](https://mlsanity.com/paper/2211.17257/) (2022)
- [Exploring LLM Prompting Strategies for Joint Essay Scoring and Feedback Generation](https://mlsanity.com/paper/2404.15845/) (2024)
- [POSIX: A Prompt Sensitivity Index For Large Language Models](https://mlsanity.com/paper/2410.02185/) (2024)
- [A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challenges](https://mlsanity.com/paper/2412.11936/) (2024)
- [Stepwise Verification and Remediation of Student Reasoning Errors with Large Language Model Tutors](https://mlsanity.com/paper/2407.09136/) (2024)
- [Nemotron-CC-Math: A 133 Billion-Token-Scale High Quality Math Pretraining Dataset](https://mlsanity.com/paper/2508.15096/) (2025)
- [Reasoning Robustness of LLMs to Adversarial Typographical Errors](https://mlsanity.com/paper/2411.05345/) (2024)
- [Towards Automated Document Revision: Grammatical Error Correction, Fluency Edits, and Beyond](https://mlsanity.com/paper/2205.11484/) (2022)
- [Learning to Make MISTAKEs: Modeling Incorrect Student Thinking And Key Errors](https://mlsanity.com/paper/2510.11502/) (2025)
- [Findings of the BEA 2025 Shared Task on Pedagogical Ability Assessment of AI-powered Tutors](https://mlsanity.com/paper/2507.10579/) (2025)
- [DrawEduMath: Evaluating Vision Language Models with Expert-Annotated Students' Hand-Drawn Math Images](https://mlsanity.com/paper/2501.14877/) (2025)
- [Pregnant Questions: The Importance of Pragmatic Awareness in Maternal Health Question Answering](https://mlsanity.com/paper/2311.09542/) (2023)
- ["Don't Forget the Teachers": Towards an Educator-Centered Understanding of Harms from Large Language Models in Education](https://mlsanity.com/paper/2502.14592/) (2025)
- [Can Vision-Language Models Evaluate Handwritten Math?](https://mlsanity.com/paper/2501.07244/) (2025)
- [Interactive Sketchpad: A Multimodal Tutoring System for Collaborative, Visual Problem-Solving](https://mlsanity.com/paper/2503.16434/) (2025)
- [Equity and Artificial Intelligence in Education: Will "AIEd" Amplify or Alleviate Inequities in Education?](https://mlsanity.com/paper/2104.12920/) (2021)
- [LLMs cannot spot math errors, even when allowed to peek into the solution](https://mlsanity.com/paper/2509.01395/) (2025)
- [Risks of AI Foundation Models in Education](https://mlsanity.com/paper/2110.10024/) (2021)
- [VEHME: A Vision-Language Model For Evaluating Handwritten Mathematics Expressions](https://mlsanity.com/paper/2510.22798/) (2025)
- [SketchMind: A Multi-Agent Cognitive Framework for Assessing Student-Drawn Scientific Sketches](https://mlsanity.com/paper/2507.22904/) (2025)
- [Teaching Models to Understand (but not Generate) High-risk Data](https://mlsanity.com/paper/2505.03052/) (2025)
- [Dead Zone of Accountability: Why Social Claims in Machine Learning Research Should Be Articulated and Defended](https://mlsanity.com/paper/2508.08739/) (2025)

## Built on by (most cited first, 1 of 1)

- [M3-ACE: Rectifying Visual Perception in Multimodal Math Reasoning via Multi-Agentic Context Engineering](https://mlsanity.com/paper/2603.08369/) (2026)

## Abstract (arXiv)

Effective mathematics education requires identifying and responding to students' mistakes. For AI to support pedagogical applications, models must perform well across different levels of student proficiency. Our work provides an extensive, year-long snapshot of how 11 vision-language models (VLMs) perform on DrawEduMath, a QA benchmark involving real students' handwritten, hand-drawn responses to math problems. We find that models' weaknesses concentrate on a core component of math education: student error. All evaluated VLMs underperform when describing work from students who require more pedagogical help, and across all QA, they struggle the most on questions related to assessing student error. Thus, while VLMs may be optimized to be math problem solving experts, our results suggest that they require alternative development incentives to adequately support educational use cases.