mlsanityData to 14 Oct 2026

SketchMind: A Multi-Agent Cognitive Framework for Assessing Student-Drawn Scientific Sketches

arXiv 2507.22904 ↗First posted 29 Jun 2025cs.HC cs.AIcited by 3

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Scientific sketches (e.g., models) offer a powerful lens into students' conceptual understanding, yet AI-powered automated assessment of such free-form, visually diverse artifacts remains a critical challenge. Existing solutions often treat sketch evaluation as either an image classification task or monolithic vision-language models, which lack interpretability, pedagogical alignment, and adaptability across cognitive levels. To address these limitations, we present SketchMind, a cognitively grounded, multi-agent framework for evaluating and improving student-drawn scientific sketches. SketchMind comprises modular agents responsible for rubric parsing, sketch perception, cognitive alignment, and iterative feedback with sketch modification, enabling personalized and transparent evaluation. We evaluate SketchMind on a curated dataset of 3,575 student-generated sketches across six science assessment items with different highest order of Bloom's level that require students to draw models to explain phenomena. Compared to baseline GPT-4o performance without SRG (average accuracy: 55.6%), and with SRG integration achieves 77.1% average accuracy (+21.4% average absolute gain). We also demonstrate that multi-agent orchestration with SRG enhances SketchMind performance, for example, GPT-4.1 gains an average 8.9% increase in sketch prediction accuracy, outperforming single-agent pipelines across all items. Human evaluators rated the feedback and co-created sketches generated by textscSketchMind with GPT-4.1, which achieved an average of 4.1 out of 5, significantly higher than those of baseline models (e.g., 2.3 for GPT-4o). Experts noted the system's potential to meaningfully support conceptual growth through guided revision. Our code and (pending approval) dataset will be released to support reproducibility and future research in AI-driven education.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 14 of its 31 references that are in the map. First posted on arXiv 29 Jun 2025.

Builds on

14 of 31 references.

References · 14

  1. 2023SKED: Sketch-guided Text-based 3D Editingcited by 73
  2. 2025Towards Zero-Shot Anomaly Detection and Reasoning with Multimodal Large Language Modelscited by 54
  3. 2025ClearSight: Visual Signal Enhancement for Object Hallucination Mitigation in Multimodal Large language Modelscited by 40
  4. 2023Sketch2Saliency: Learning to Detect Salient Objects from Human Drawingscited by 17
  5. 2025DrawEduMath: Evaluating Vision Language Models with Expert-Annotated Students' Hand-Drawn Math Imagescited by 17
  6. 2024Using Generative AI and Multi-Agents to Provide Automatic Feedbackcited by 15
  7. 2023SEVA: Leveraging sketches to evaluate alignment between human and machine visual abstractioncited by 14
  8. 2023NERIF: GPT-4V for Automatic Scoring of Drawn Modelscited by 13
  9. 2025Evolution of AI in Education: Agentic Workflowscited by 13
  10. 2023Gemini Pro Defeated by GPT-4V: Evidence from Educationcited by 12
  11. 2023SketchXAI: A First Look at Explainability for Human Sketchescited by 10
  12. 2025Assessing AI-Generated Questions' Alignment with Cognitive Frameworks in Educational Assessmentcited by 8
2 more
  1. 2023Data-Free Sketch-Based Image Retrievalcited by 7
  2. 2024"My Grade is Wrong!": A Contestable AI Framework for Interactive Feedback in Evaluating Student Essayscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026SketchJudge: A Diagnostic Benchmark for Grading Hand-drawn Diagrams with Multimodal Large Language Models
  2. 2025SketchAgent: Generating Structured Diagrams from Hand-Drawn Sketches
  3. 2026Exploring Agentic Workflows for Generating High Quality Math Visual Aids
  4. 2026Sketch2Feedback: Grammar-in-the-Loop Framework for Rubric-Aligned Feedback on Student STEM Diagrams
  5. 2025Proactive Agentic Whiteboards: Enhancing Diagrammatic Learning
  6. 2025See it. Say it. Sorted: Agentic System for Compositional Diagram Generation
  7. 2025From EduVisBench to EduVisAgent: A Benchmark and Multi-Agent Framework for Reasoning-Driven Pedagogical Visualization
  8. 2026PainterBench: A Figural Divergent-Thinking Benchmark for Tool-Using Language Models