SketchMind: A Multi-Agent Cognitive Framework for Assessing Student-Drawn Scientific Sketches
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
Scientific sketches (e.g., models) offer a powerful lens into students' conceptual understanding, yet AI-powered automated assessment of such free-form, visually diverse artifacts remains a critical challenge. Existing solutions often treat sketch evaluation as either an image classification task or monolithic vision-language models, which lack interpretability, pedagogical alignment, and adaptability across cognitive levels. To address these limitations, we present SketchMind, a cognitively grounded, multi-agent framework for evaluating and improving student-drawn scientific sketches. SketchMind comprises modular agents responsible for rubric parsing, sketch perception, cognitive alignment, and iterative feedback with sketch modification, enabling personalized and transparent evaluation. We evaluate SketchMind on a curated dataset of 3,575 student-generated sketches across six science assessment items with different highest order of Bloom's level that require students to draw models to explain phenomena. Compared to baseline GPT-4o performance without SRG (average accuracy: 55.6%), and with SRG integration achieves 77.1% average accuracy (+21.4% average absolute gain). We also demonstrate that multi-agent orchestration with SRG enhances SketchMind performance, for example, GPT-4.1 gains an average 8.9% increase in sketch prediction accuracy, outperforming single-agent pipelines across all items. Human evaluators rated the feedback and co-created sketches generated by textscSketchMind with GPT-4.1, which achieved an average of 4.1 out of 5, significantly higher than those of baseline models (e.g., 2.3 for GPT-4o). Experts noted the system's potential to meaningfully support conceptual growth through guided revision. Our code and (pending approval) dataset will be released to support reproducibility and future research in AI-driven education.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 14 of its 31 references that are in the map. First posted on arXiv 29 Jun 2025.
Builds on
14 of 31 references.
References · 14
- 2023SKED: Sketch-guided Text-based 3D Editingcited by 73
- 2025Towards Zero-Shot Anomaly Detection and Reasoning with Multimodal Large Language Modelscited by 54
- 2025ClearSight: Visual Signal Enhancement for Object Hallucination Mitigation in Multimodal Large language Modelscited by 40
- 2023Sketch2Saliency: Learning to Detect Salient Objects from Human Drawingscited by 17
- 2025DrawEduMath: Evaluating Vision Language Models with Expert-Annotated Students' Hand-Drawn Math Imagescited by 17
- 2024Using Generative AI and Multi-Agents to Provide Automatic Feedbackcited by 15
- 2023SEVA: Leveraging sketches to evaluate alignment between human and machine visual abstractioncited by 14
- 2023NERIF: GPT-4V for Automatic Scoring of Drawn Modelscited by 13
- 2025Evolution of AI in Education: Agentic Workflowscited by 13
- 2023Gemini Pro Defeated by GPT-4V: Evidence from Educationcited by 12
- 2023SketchXAI: A First Look at Explainability for Human Sketchescited by 10
- 2025Assessing AI-Generated Questions' Alignment with Cognitive Frameworks in Educational Assessmentcited by 8
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026The Aftermath of DrawEduMath: Vision Language Models Underperform with Struggling Students and Misdiagnose Errorscited by 1
- 2026SketchVLM: Vision language models can annotate images to explain thoughts and guide userscited by 0
- 2026Simulating Validity: Modal Decoupling in MLLM Generated Feedback on Science Drawingscited by 0
- 2026SketchVLM: Vision language models can annotate images to explain thoughts and guide users23 Apr 2026
- 2026Simulating Validity: Modal Decoupling in MLLM Generated Feedback on Science Drawings5 Apr 2026
- 2026The Aftermath of DrawEduMath: Vision Language Models Underperform with Struggling Students and Misdiagnose Errors1 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2026SketchJudge: A Diagnostic Benchmark for Grading Hand-drawn Diagrams with Multimodal Large Language Models
- 2025SketchAgent: Generating Structured Diagrams from Hand-Drawn Sketches
- 2026Exploring Agentic Workflows for Generating High Quality Math Visual Aids
- 2026Sketch2Feedback: Grammar-in-the-Loop Framework for Rubric-Aligned Feedback on Student STEM Diagrams
- 2025Proactive Agentic Whiteboards: Enhancing Diagrammatic Learning
- 2025See it. Say it. Sorted: Agentic System for Compositional Diagram Generation
- 2025From EduVisBench to EduVisAgent: A Benchmark and Multi-Agent Framework for Reasoning-Driven Pedagogical Visualization
- 2026PainterBench: A Figural Divergent-Thinking Benchmark for Tool-Using Language Models