# CSTutorBench: Benchmarking Small Language Models as Tutors for Block-Based Programming

arXiv: https://arxiv.org/abs/2607.05571 · HTML: https://mlsanity.com/paper/2607.05571/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 21 references that are in the map.
- First posted on arXiv 6 Jul 2026.

## Builds on (most cited first, 5 of 5)

- [A Survey on Large Language Models for Code Generation](https://mlsanity.com/paper/2406.00515/) (2024)
- [EduBench: A Comprehensive Benchmarking Dataset for Evaluating Large Language Models in Diverse Educational Scenarios](https://mlsanity.com/paper/2505.16160/) (2025)
- [TutorBench: A Benchmark To Assess Tutoring Capabilities Of Large Language Models](https://mlsanity.com/paper/2510.02663/) (2025)
- [Towards a Human-in-the-Loop Framework for Reliable Patch Evaluation Using an LLM-as-a-Judge](https://mlsanity.com/paper/2511.10865/) (2025)
- [From Answer Givers to Design Mentors: Guiding LLMs with the Cognitive Apprenticeship Model](https://mlsanity.com/paper/2601.19053/) (2026)

## Abstract (arXiv)

Large language models are increasingly explored as AI tutors, yet deploying them in K-12 settings raises concerns around privacy, cost, and reliance on proprietary models. Small language models (SLMs) offer a promising alternative, but selecting the right model for a specific educational context remains difficult, particularly when the target domain, such as block-based programming, is largely absent from model training data. We introduce CSTutorBench, a benchmark for evaluating language models as CS tutors in VEX VR, a block-based robotics environment. The benchmark comprises 17 scenario-based questions scored against a pedagogical rubric grounded in established tutoring and feedback research, with a human-in-the-loop LLM-as-judge pipeline for evaluation. Preliminary findings across 11 models (4B-120B parameters) reveal that models perform well on surface-level criteria such as vocabulary and tone but struggle with deeper pedagogical behaviors, particularly avoiding answer leakage and engaging with student debugging histories. In our sample, model family and instruction-tuning approach appear to be better predictors of tutoring quality than parameter count alone, though the small number of models limits the strength of this conclusion. A targeted prompt revision grounded in recent educational prompt engineering research improved scores for 10 of 11 models. These results underscore the value of context-specific, pedagogically grounded benchmarks for SLM selection in educational deployment.