# SketchMind: A Multi-Agent Cognitive Framework for Assessing Student-Drawn Scientific Sketches

arXiv: https://arxiv.org/abs/2507.22904 · HTML: https://mlsanity.com/paper/2507.22904/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use.
- Builds on 14 of its 31 references that are in the map.
- First posted on arXiv 29 Jun 2025.

## Placed in

- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 14 of 14)

- [SKED: Sketch-guided Text-based 3D Editing](https://mlsanity.com/paper/2303.10735/) (2023)
- [Towards Zero-Shot Anomaly Detection and Reasoning with Multimodal Large Language Models](https://mlsanity.com/paper/2502.07601/) (2025)
- [ClearSight: Visual Signal Enhancement for Object Hallucination Mitigation in Multimodal Large language Models](https://mlsanity.com/paper/2503.13107/) (2025)
- [Sketch2Saliency: Learning to Detect Salient Objects from Human Drawings](https://mlsanity.com/paper/2303.11502/) (2023)
- [DrawEduMath: Evaluating Vision Language Models with Expert-Annotated Students' Hand-Drawn Math Images](https://mlsanity.com/paper/2501.14877/) (2025)
- [Using Generative AI and Multi-Agents to Provide Automatic Feedback](https://mlsanity.com/paper/2411.07407/) (2024)
- [SEVA: Leveraging sketches to evaluate alignment between human and machine visual abstraction](https://mlsanity.com/paper/2312.03035/) (2023)
- [NERIF: GPT-4V for Automatic Scoring of Drawn Models](https://mlsanity.com/paper/2311.12990/) (2023)
- [Evolution of AI in Education: Agentic Workflows](https://mlsanity.com/paper/2504.20082/) (2025)
- [Gemini Pro Defeated by GPT-4V: Evidence from Education](https://mlsanity.com/paper/2401.08660/) (2023)
- [SketchXAI: A First Look at Explainability for Human Sketches](https://mlsanity.com/paper/2304.11744/) (2023)
- [Assessing AI-Generated Questions' Alignment with Cognitive Frameworks in Educational Assessment](https://mlsanity.com/paper/2504.14232/) (2025)
- [Data-Free Sketch-Based Image Retrieval](https://mlsanity.com/paper/2303.07775/) (2023)
- ["My Grade is Wrong!": A Contestable AI Framework for Interactive Feedback in Evaluating Student Essays](https://mlsanity.com/paper/2409.07453/) (2024)

## Built on by (most cited first, 3 of 3)

- [The Aftermath of DrawEduMath: Vision Language Models Underperform with Struggling Students and Misdiagnose Errors](https://mlsanity.com/paper/2603.00925/) (2026)
- [SketchVLM: Vision language models can annotate images to explain thoughts and guide users](https://mlsanity.com/paper/2604.22875/) (2026)
- [Simulating Validity: Modal Decoupling in MLLM Generated Feedback on Science Drawings](https://mlsanity.com/paper/2604.26957/) (2026)

## Abstract (arXiv)

Scientific sketches (e.g., models) offer a powerful lens into students' conceptual understanding, yet AI-powered automated assessment of such free-form, visually diverse artifacts remains a critical challenge. Existing solutions often treat sketch evaluation as either an image classification task or monolithic vision-language models, which lack interpretability, pedagogical alignment, and adaptability across cognitive levels. To address these limitations, we present SketchMind, a cognitively grounded, multi-agent framework for evaluating and improving student-drawn scientific sketches. SketchMind comprises modular agents responsible for rubric parsing, sketch perception, cognitive alignment, and iterative feedback with sketch modification, enabling personalized and transparent evaluation. We evaluate SketchMind on a curated dataset of 3,575 student-generated sketches across six science assessment items with different highest order of Bloom's level that require students to draw models to explain phenomena. Compared to baseline GPT-4o performance without SRG (average accuracy: 55.6%), and with SRG integration achieves 77.1% average accuracy (+21.4% average absolute gain). We also demonstrate that multi-agent orchestration with SRG enhances SketchMind performance, for example, GPT-4.1 gains an average 8.9% increase in sketch prediction accuracy, outperforming single-agent pipelines across all items. Human evaluators rated the feedback and co-created sketches generated by textscSketchMind with GPT-4.1, which achieved an average of 4.1 out of 5, significantly higher than those of baseline models (e.g., 2.3 for GPT-4o). Experts noted the system's potential to meaningfully support conceptual growth through guided revision. Our code and (pending approval) dataset will be released to support reproducibility and future research in AI-driven education.