mlsanityData to 14 Oct 2026

Vision-Language Models for Criterion-Level Grading of Handwritten Examinations in Outcome-Based Education

arXiv 2609.14284 ↗First posted 13 Sep 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Criterion-level grading connects examination performance to learning outcomes, but manual marking introduces workload and variation between markers. This study evaluates vision-language models (VLMs) for handwritten outcome-based assessment across five dimensions: accuracy, human agreement, repeated-run reliability, error concentration, and explanation quality. Using 1,982 criterion-level records from 485 undergraduate examination answers, we compare 20 configurations spanning Qwen2.5-VL, InternVL3, Pixtral, a Donut baseline, and a cascade ensemble. Evaluation setups include zero-shot prompting, few-shot prompting, partial fine-tuning, and Low-Rank Adaptation (LoRA). Two independent faculty markers regraded all 291 test criteria, providing a human agreement baseline on the same assessment materials. Qwen2.5-VL with LoRA achieved Quadratic Weighted Kappa (QWK) of 0.727 and mean absolute error of 0.435 marks against the examiner, compared with mean human-pair QWK of 0.551. This comparison reflects calibration to the examiner's training marks. LoRA outperformed partial fine-tuning for all three instruction-tuned VLMs, while few-shot prompting reduced QWK in every configuration with valid prompted scores. Aggregate reliability and exact repeatability diverged: intraclass correlations ranged from 0.790 to 0.874, yet 50.2-63.6% of criteria changed marks across five sampled runs. Attention-guided deletion showed no statistically significant advantage over random masking, and four faculty reviewers reached no consensus on explanation usefulness. These findings highlight the need for rubric-specific calibration, repeatable scoring, review of consequential errors, and separate validation of explanations. The released evaluation protocol supports criterion-level assessment research and grading tools with teacher oversight.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 19 of its 46 references that are in the map. First posted on arXiv 13 Sep 2026.

Builds on

19 of 46 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation28 Jan

Other references · 18

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2025Qwen2.5-VL Technical Reportcited by 5,587
  6. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  7. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  8. 2020Quantifying Attention Flow in Transformerscited by 775
  9. 2020Transformer Interpretability Beyond Attention Visualizationcited by 627
  10. 2021OCR-free Document Understanding Transformercited by 407
  11. 2024LoRA Learns Less and Forgets Lesscited by 328
  12. 2021TrOCR: Transformer-based Optical Character Recognition with Pre-trained Modelscited by 279
6 more
  1. 2024MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizationscited by 183
  2. 2024Pixtral 12Bcited by 179
  3. 2018Automated essay scoring with string kernels and word embeddingscited by 31
  4. 2024On the Faithfulness of Vision Transformer Explanationscited by 19
  5. 2025Benchmarking Large Language Models for Handwritten Text Recognitioncited by 11
  6. 2026Multimodal examination answer data with expert-designed Outcome-Based Education rubrics for criterion-level assessmentcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Evaluating Vision-Language and Large Language Models for Automated Student Assessment in Indonesian Classrooms
  2. 2026When VLMs 'Fix' Students: Identifying and Penalizing Over-Correction in the Evaluation of Multi-line Handwritten Math OCR
  3. 2026Aesthetic Assessment of Chinese Handwritings Based on Vision Language Models
  4. 2026Towards Fully Automated Exam Grading: Fairness-Aware Recognition of Handwritten Answers with Foundation Models
  5. 2025Seeing the Big Picture: Evaluating Multimodal LLMs' Ability to Interpret and Grade Handwritten Student Work
  6. 2026Bounding Boxes to Improve Small Language Model Performance on Vision-Based Grading Tasks
  7. 2025Can Vision-Language Models Evaluate Handwritten Math?
  8. 2025DrawEduMath: Evaluating Vision Language Models with Expert-Annotated Students' Hand-Drawn Math Images