# Learnable Assessment Skills for LLM-based Automated Scoring: Rubric Construction via Iterative Optimization

arXiv: https://arxiv.org/abs/2605.29274 · HTML: https://mlsanity.com/paper/2605.29274/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 19 references that are in the map.
- First posted on arXiv 28 May 2026.

## Builds on (most cited first, 15 of 15)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [ExpeL: LLM Agents Are Experiential Learners](https://mlsanity.com/paper/2308.10144/) (2023)
- [GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning](https://mlsanity.com/paper/2507.19457/) (2025)
- [Inducing Programmatic Skills for Agentic Tasks](https://mlsanity.com/paper/2504.06821/) (2025)
- [Feedback Descent: Open-Ended Text Optimization via Pairwise Comparison](https://mlsanity.com/paper/2511.07919/) (2025)
- [Grade Like a Human: Rethinking Automated Assessment with Large Language Models](https://mlsanity.com/paper/2405.19694/) (2024)
- [A LLM-Powered Automatic Grading Framework with Human-Level Guidelines Optimization](https://mlsanity.com/paper/2410.02165/) (2024)
- [AutoSCORE: Enhancing Automated Scoring with Multi-Agent Large Language Models via Structured Component Recognition](https://mlsanity.com/paper/2509.21910/) (2025)
- [Using Large Language Models for Automated Grading of Student Writing about Science](https://mlsanity.com/paper/2412.18719/) (2024)
- [AGIBench: A Multi-granularity, Multimodal, Human-referenced, Auto-scoring Benchmark for Large Language Models](https://mlsanity.com/paper/2309.06495/) (2023)
- [Confusion-Aware Rubric Optimization for LLM-based Automated Grading](https://mlsanity.com/paper/2603.00451/) (2026)
- [Designing Reliable LLM-Assisted Rubric Scoring for Constructed Responses: Evidence from Physics Exams](https://mlsanity.com/paper/2604.12227/) (2026)
- [Using Learning Progressions to Guide AI Feedback for Science Learning](https://mlsanity.com/paper/2603.03249/) (2026)

## Abstract (arXiv)

LLM-based automated scoring approaches near-human performance, but scaling to new tasks remains bottlenecked by the per-item human configuration of upstream stages such as rubric construction. Human experts bypass this bottleneck through evaluation heuristics developed over extensive practice. We ask whether LLMs can learn similar heuristics directly from scoring experience, and formalize this as the concept of assessment skills: item-independent natural-language procedural knowledge that guides LLMs through specific stages of the scoring workflow. Focusing on rubric construction as a first instantiation, we propose an iterative framework that decomposes a skill into a fixed scaffold and learnable item-agnostic rules, refining the rules through LLM-driven diagnosis of scoring errors and validation-gated selection. The framework requires no expert-written rubric. On all ten ASAP-SAS items, optimized skills substantially improve LLM-based scoring and frequently surpass the dataset-provided expert rubric. Cross-item transfer experiments further reveal that learned skills capture both generalizable and item-specific patterns.