# Artificial-Intelligence Grading Assistance for Handwritten Components of a Calculus Exam

arXiv: https://arxiv.org/abs/2510.05162 · HTML: https://mlsanity.com/paper/2510.05162/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 43 references that are in the map.
- First posted on arXiv 4 Oct 2025.

## Builds on (most cited first, 3 of 3)

- [A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores](https://mlsanity.com/paper/2002.08035/) (2020)
- [Performance of the Pre-Trained Large Language Model GPT-4 on Automated Short Answer Grading](https://mlsanity.com/paper/2309.09338/) (2023)
- [Assisting the Grading of a Handwritten General Chemistry Exam with Artificial Intelligence](https://mlsanity.com/paper/2509.10591/) (2025)

## Built on by (most cited first, 3 of 3)

- [Automated Grading of Handwritten Mathematics Using Vision-Capable LLMs](https://mlsanity.com/paper/2605.19043/) (2026)
- [Large Scale AI Grading of Handwritten Physics Assessments: Score Agreement and Olympiad Team Selection Outcomes](https://mlsanity.com/paper/2608.20521/) (2026)
- [SoK: Grading Open-Text Legal Answers, from Law School Exams to Trustworthy Rubric Judges](https://mlsanity.com/paper/2610.19992/) (2026)

## Abstract (arXiv)

We investigate whether contemporary multimodal LLMs can assist with grading open-ended calculus at scale without eroding validity. In a large first-year exam, students' handwritten work was graded by GPT-5 against the same rubric used by teaching assistants (TAs), with fractional credit permitted; TA rubric decisions served as ground truth. We calibrated a human-in-the-loop filter that combines a partial-credit threshold with an Item Response Theory (2PL) risk measure based on the deviation between the AI score and the model-expected score for each student-item. Unfiltered AI-TA agreement was moderate, adequate for low-stakes feedback but not for high-stakes use. Confidence filtering made the workload-quality trade-off explicit: under stricter settings, AI delivered human-level accuracy, but also left roughly 70% of the items to be graded by humans. Psychometric patterns were constrained by low stakes on the open-ended portion, a small set of rubric checkpoints, and occasional misalignment between designated answer regions and where work appeared. Practical adjustments such as slightly higher weight and protected time, a few rubric-visible substeps, stronger spatial anchoring should raise ceiling performance. Overall, calibrated confidence and conservative routing enable AI to reliably handle a sizable subset of routine cases while reserving expert judgment for ambiguous or pedagogically rich responses.