# Evaluating GPT-4 at Grading Handwritten Solutions in Math Exams

arXiv: https://arxiv.org/abs/2411.05231 · HTML: https://mlsanity.com/paper/2411.05231/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 6 references that are in the map.
- First posted on arXiv 7 Nov 2024.

## Builds on (most cited first, 2 of 2)

- [Exploring LLM Prompting Strategies for Joint Essay Scoring and Feedback Generation](https://mlsanity.com/paper/2404.15845/) (2024)
- [Automatic Short Math Answer Grading via In-context Meta-learning](https://mlsanity.com/paper/2205.15219/) (2022)

## Built on by (most cited first, 10 of 10)

- [StepMathAgent: A Step-Wise Agent for Evaluating Mathematical Processes through Tree-of-Error](https://mlsanity.com/paper/2503.10105/) (2025)
- [Automated Grading of Students' Handwritten Graphs: A Comparison of Meta-Learning and Vision-Large Language Models](https://mlsanity.com/paper/2507.03056/) (2025)
- [Evaluating AI Grading on Real-World Handwritten College Mathematics: A Large-Scale Study Toward a Benchmark](https://mlsanity.com/paper/2603.00895/) (2026)
- [Human-in-the-Loop LLM Grading for Handwritten Mathematics Assessments](https://mlsanity.com/paper/2603.13083/) (2026)
- [Generative AI performance in core undergraduate mathematics: a curriculum-level case study](https://mlsanity.com/paper/2509.13359/) (2025)
- [Grading Handwritten Engineering Exams with Multimodal Large Language Models](https://mlsanity.com/paper/2601.00730/) (2026)
- [Can MLLMs Read Students' Minds? Unpacking Multimodal Error Analysis in Handwritten Math](https://mlsanity.com/paper/2603.24961/) (2026)
- [Is Mathematical Problem-Solving Expertise in Large Language Models Associated with Assessment Performance?](https://mlsanity.com/paper/2603.25633/) (2026)
- [LLM Performance on a Real, Double-Marked GCSE Benchmark](https://mlsanity.com/paper/2606.24973/) (2026)
- [A Human-in-the-Loop Framework for AI-Assisted Scoring in Large-Scale Writing Assessment](https://mlsanity.com/paper/2609.05143/) (2026)

## Abstract (arXiv)

Recent advances in generative artificial intelligence (AI) have shown promise in accurately grading open-ended student responses. However, few prior works have explored grading handwritten responses due to a lack of data and the challenge of combining visual and textual information. In this work, we leverage state-of-the-art multi-modal AI models, in particular GPT-4o, to automatically grade handwritten responses to college-level math exams. Using real student responses to questions in a probability theory exam, we evaluate GPT-4o's alignment with ground-truth scores from human graders using various prompting techniques. We find that while providing rubrics improves alignment, the model's overall accuracy is still too low for real-world settings, showing there is significant room for growth in this task.