# Small, Private Language Models as Teammates for Educational Assessment Design

arXiv: https://arxiv.org/abs/2605.15015 · HTML: https://mlsanity.com/paper/2605.15015/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 37 references that are in the map.
- First posted on arXiv 14 May 2026.

## Builds on (most cited first, 11 of 11)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://mlsanity.com/paper/2009.07118/) (2020)
- [Enhancing LLM-Based Feedback: Insights from Intelligent Tutoring Systems and the Learning Sciences](https://mlsanity.com/paper/2405.04645/) (2024)
- [Automated Educational Question Generation at Different Bloom's Skill Levels using Large Language Models: Strategies and Evaluation](https://mlsanity.com/paper/2408.04394/) (2024)
- [How Useful are Educational Questions Generated by Large Language Models?](https://mlsanity.com/paper/2304.06638/) (2023)
- [Scalable Educational Question Generation with Pre-trained Language Models](https://mlsanity.com/paper/2305.07871/) (2023)
- [Can Large Language Models Match Tutoring System Adaptivity? A Benchmarking Study](https://mlsanity.com/paper/2504.05570/) (2025)
- [Exploring the Capabilities of Prompted Large Language Models in Educational and Assessment Applications](https://mlsanity.com/paper/2405.11579/) (2024)
- [GPT-4 as a Homework Tutor can Improve Student Engagement and Learning Outcomes](https://mlsanity.com/paper/2409.15981/) (2024)
- [From Cloud to Edge: Rethinking Generative AI for Low-Resource Design Challenges](https://mlsanity.com/paper/2402.12702/) (2024)

## Built on by (most cited first, 2 of 2)

- [Bounding Boxes to Improve Small Language Model Performance on Vision-Based Grading Tasks](https://mlsanity.com/paper/2607.18767/) (2026)
- [A Benchmark for LLM's Understanding of Middle School and High School Science Topics](https://mlsanity.com/paper/2609.32020/) (2026)

## Abstract (arXiv)

Generative AI increasingly supports educational design tasks, e.g., through Large Language Models (LLMs), demonstrating the capability to design assessment questions that are aligned with pedagogical frameworks (e.g., Bloom's taxonomy). However, they often rely on subjective or limited evaluation methods; focus primarily on proprietary models; or rarely systematically examine generation, evaluation, or deployment constraints in real educational settings. Meanwhile, Small Language Models (SLMs) have emerged as local alternatives that better address privacy and resource limitations; yet their effectiveness for assessment tasks remains underexplored. To address this gap, we systematically compare LLMs and SLMs for assessment question design; evaluate generation quality across Bloom's taxonomy levels using reproducible, pedagogically grounded metrics; and further assess model-based judging against expert-informed evaluation by analyzing reliability and agreement patterns. Results show that SLMs achieve competitive performance across key pedagogically motivated quality dimensions while enabling local, privacy-sensitive deployment. However, model-based evaluations also exhibit systematic inconsistencies and bias relative to expert ratings. These findings provide evidence to posit language models as bounded assistants in assessment workflows; underscore the necessity of Human-in-the-Loop; and advance the automated educational question generation field by examining quality, reliability, and deployment-aware trade-offs.